The Czechs say the 30-square kilometre Turów lignite mine is damaging nearby Czech border villages and have pushed their case to the highest European Union court, prompting the talks on a deal.
After months of negotiations, Warsaw brought an offer to Prague in the past week to end the spat between the central European allies.
Czech Environment Minister Richard Brabec told reporters in a late evening televised briefing on Thursday that the sides had "for now" not reached a final agreement.
He said a main reason was over different opinions on the length of the agreement, with the Czech side pushing for a long-term framework.
Czech Television, though, reported no times for further talks had been set and that the Polish delegation was returning home. It quoted Polish Climate Minister Michał Kurtyka as saying a "very good offer" was not accepted.
Talks, running since May, have sought to clarify financial compensation and technical improvements to safeguard the environment around the mine. Czech communities have complained of reduced water supplies because of the mining.
Turów is slated to run until 2044 and feeds an adjoining power plant that covers as much as 7% of Poland's energy output.
Warsaw says it is vital for jobs and energy.
Poland has so far defied a Court of Justice of the European Union (CJEU) order from May to halt mining at Turów while the Czech case continued. The court last week ordered Warsaw to pay a 500,000 euro ($579,000) daily penalty.
(Reporting by Jason Hovet; editing by Grant McCool)
Source: Reuters