IMMIGRATION Minister Brendan O'Connor has urged his Labor colleagues to "get on with the job of governing" as a fresh round of leadership speculation broke out at the weekend.
Media reports during the past three days suggested an increasing number of panicked Labor MPs were considering a switch back to former prime minister Kevin Rudd, although such a move appeared unlikely at this late stage of the election cycle.
As many as 32 Labor MPs have made official requests for Mr Rudd to campaign in their seats between now and the election.
On Sunday, respected political journalist and commentator Barrie Cassidy suggested support for Julia Gillard was collapsing, and on Monday there were reports the PM's most powerful ally, Bill Shorten, was under mounting pressure from within the party to call on her to resign.
Despite increased public appearances in the past week, Mr Rudd again denied on Monday he was making a final run at the leadership.
It came as leaked internal party polling revealed Labor faced losing up to 40 seats, including nine held by ministers. Newspoll and Nielsen are due to release polls next week.
But Mr O'Connor said people were sick of the party talking about itself and expected it to "focus on issues that matter".
"They want us delivering to all Australians," Mr O'Connor said on Sunrise.
"But speculation may continue. The only challenge to the Prime Minister's job is Tony Abbott, and he has no plan for Australia's future."
With just two caucus meetings left before the election - the first next Tuesday - time is running out for Labor to install Mr Rudd.
But at least one of the key independents keeping Labor in power said his agreement with the party would be "null and void" if Ms Gillard was deposed.
Not for the first time Tony Windsor said the agreement he forged with Labor in the days after the 2010 was with Ms Gillard and no one else.
"If she disappears for some reason there is no arrangement," Mr Windsor told ABC radio.
If Labor was to lose the support of the independents it could take Australians to the polls at least a month earlier than expected.