In 1866, Ballance and his wife migrated to New Zealand, intending to start in business there as a small jeweller. After settling at Wanganui, however, he took an opportunity which soon arose to found a newspaper, the Wanganui Herald. He became the paper's editor, and remained chief owner for the rest of his life. During the fighting with the Maori chief Titokowaru, in 1867, Ballance was involved in the raising of a volunteer cavalry troop, in which he received a commission. He was later deprived of this owing to the appearance in his newspaper of articles criticizing the management of the campaign. He had, however, behaved well in the field, and, in spite of his dismissal, was awarded the New Zealand war medal.

In the period following the conflict, Ballance's status in Wanganui grew. He was respected for his management of the Wanganui Herald, particularly his forthright and direct approach to reporting. He also became increasingly involved in the affairs of the town itself, establishing a number of societies and associations. Perhaps the least important to Wanganui, but among the most important to Ballance himself, was the chess club - Ballance became a skilled player of the game. In 1868, Ballance's wife died of illness, aged only twenty-four. Two years later, he married Ellen Anderson, daughter of a Wellington architect.

In 1875, Ballance entered parliament, having stood in the electorate of Rangitikei. He campaigned on two major issues - the abolition of the provinces (widely regarded as incompetent, petty, and obstructive) and the provision of free education. In 1887, he entered the cabinet of George Grey, a former Governor who was then serving as Premier. Grey's policies were not closely aligned with those of Ballance, but Ballance believed that he could nevertheless accomplish something worthwhile. He served as minister of customs, as minister of education, and later as treasurer. His alliance with Grey ended with a notorious and very painful quarrel, however - Ballance found Grey far too controlling and authoritarian.

In 1873, Ballance became a minister in the cabinet of Robert Stout, a fellow liberal. He was made Minister of Lands and Immigration, Minister of Defence, and Minister of Native Affairs (relations with the Maori). In his role as Minister of Lands, he encouraged intensive settlement of rural areas, aiming to increase the number of people leaving the cities to "work the land" (a measure he believed was essential to increase productivity and self-sufficiency). His system of state-aided ?village settlements,? by which small holdings were leased by the crown to farmers, and money lent them to make a beginning of building and cultivation, was generally successful.

Despite this desire for increased settlement of colonist-held lands, however, he strongly supported the rights of Maori to retain the land they still held - many other politicians of his time believed that acquisition of Maori land was essential to increasing settlement. He also reduced the government's military presence in areas where strong tensions with Maori existed, and made an attempt to familiarize himself with Maori language and culture. In 1887, Stout's government lost the general elections, but Ballance himself remained popular. Illness initially prevented his full participation in politics, but with his recovery in July 1889, he became Leader of the Opposition.

In 1890, Ballance led a loose coalition of liberal politicians to victory in the general election. Harry Atkinson, the Premier who had defeated Stout, was forced to resign, but not before stacking the Legislative Council with his own supporters. This was to prove a serious problem for Ballance's premiership, but not one which he was unable to overcome (partly by reducing the life-tenure of legislative councillors to one of seven years). His successful battle with the Governor over changes to the Legislative Council also helped define the relationship between the elected Premier and the appointed Governor (mostly in the Premier's favour).

During his term, Ballance was actively concerned in the advocacy of women's suffrage, declaring to parliament that he believed in the "absolute equality of the sexes." This was a cause he had partially inherited from his colleague in the Stout premiership, Julius Vogel, and in which he was influenced by his politically astute wife, Ellen. Ballance was also responsible for the establishment, in 1891, of the progressive land tax and progressive income tax. He was widely praised for his handling of the economy, which expanded greatly during his term.

As leader of parliament's liberal faction, he brought his allies and colleagues into the Liberal Party, New Zealand's first political party. The alliance was intended to embody the liberal ideas of Stout, Vogel, and Ballance himself. The next four premiers of New Zealand were to be from the Liberal Party, although some (such as Richard Seddon) did not live up to the ideals that Ballance tried to establish.

Quiet and unassuming in manner, Ballance, who was a well-read man, always seemed fonder of his books and his chessboard than of public bustle. He has been described as "unassuming and unpretentious", and was known to be quiet, polite, and extremely patient.

In 1893, at the height of his success and popularity, he died in Wellington of an intestinal disease after a severe surgical operation. Ballance is believed to have supported Robert Stout as his successor, but his rapid descent into illness prevented him from securing that outcome. Instead, he was followed as Premier by Richard Seddon. A statue was erected to Ballance's memory in front of Parliament House, Wellington.