Oliver told a business audience in Toronto on Monday that there won't be any big stimulus package next year.
But he said once the budget is balanced and moves into a surplus position there will be relief for "hard-working Canadian families" who pay too much in taxes.
Oliver didn't address the contentious issue of income splitting and he stressed that job creation would continue to be a top priority.
Oliver, a lawyer with an MBA who worked in the private sector finance community before entering politics, was addressing the Bay Street crowd for the first time since taking over as finance minister from Jim Flaherty last month.
Until March, Flaherty and had been the Harper government's only finance minister and was the one who managed the economy through the 2008-2009 recession.
In the past, Oliver has said he would carry on his predecessor's focus on creating jobs , economic growth and balancing the budget.
However, he has also appeared to back Prime Minister Stephen Harper on income splitting, a 2011 election promise that Flaherty had publicly questioned.
He has also differed with Flaherty's approach to mortgage rates, recently saying he would not intervene after the Bank of Montreal announced a half-point cut to its key mortgage rate, dropping its fixed, five-year setting to 2.99 per cent. Flaherty chided BMO for such a move last year, warning that such low rates could overheat the housing market.