The 7.5-magnitude quake, which hit Ishikawa prefecture on the main island of Honshu, triggered tsunami waves more than a metre high, toppled buildings, caused a major fire and tore apart roads.
As daylight arrived, the scale of the destruction on the Noto Peninsula emerged with buildings still smouldering, houses flattened, fishing boats sunk or washed ashore and highways hit by landslides.
"It was such a powerful jolt," Tsugumasa Mihara, 73, told AFP as he queued with hundreds of others for water in the town of Shika.
"What a terrible way to start the year," he said.
Police said six people had been killed although the toll was almost certain to climb. The Kyodo news agency reported that 24 people had died, including 15 in the badly hit port city of Wajima.
"Very extensive damage has been confirmed, including numerous casualties, building collapses and fires," Prime Minister Fumio Kishida said after a disaster response meeting.
"We have to race against time to search for and rescue victims of the disaster."
Aerial news footage showed devastation from the major fire in Wajima, where a seven-storey commercial building collapsed.
Almost 45,000 households were without power in the region, which saw temperatures touch freezing overnight, the local energy provider said. Many cities were without running water.
The US Geological Survey (USGS) said the quake had a magnitude of 7.5. Japan's meteorological agency measured it at 7.6, and said it was one of more than 150 to shake the region through Tuesday morning.
Several strong jolts were felt early Tuesday, including one measuring 5.6 that prompted national broadcaster NHK to switch to a special programme.
"Please take deep breaths," the presenter said, reminding viewers to check for fires in their kitchens.
On Monday, waves at least 1.2 metres (four feet) high hit Wajima and a series of smaller tsunamis were reported elsewhere.
Warnings of much larger waves proved unfounded and on Tuesday Japan lifted all tsunami warnings.
Images on social media showed cars and houses in Ishikawa shaking violently and terrified people cowering in shops and train stations. Houses collapsed and huge cracks appeared in roads.
A team of firefighters crawled under a collapsed commercial building in Wajima, television footage showed.
"Hang in there! Hang in there," they shouted as they battled through piles of wooden beams with an electric saw.
"There was shaking like I have never experienced before," an elderly man told NHK.
"Inside my house, it was so terrible... I am still alive. Maybe I have to be content with that."
The fire in Wajima engulfed dozens of structures, video footage showed, with people being evacuated in the dark, some with blankets and others carrying babies.
NHK reported that 25 houses had collapsed in the city, including 14 that may have had people trapped inside.
A duty officer at Wajima Fire Department said authorities were still being overwhelmed Tuesday by rescue requests and reports of damage.
Ishikawa Governor Hiroshi Hase wrote on X, formerly Twitter, that roads have been cut in widespread areas by landslides or cracking, while in the port of Suzu "multiple" vessels had capsized.
A total of 62,000 people had been ordered to evacuate, according to the fire and disaster management agency.
About 1,000 people were staying at a military base, the defence ministry said.
Defense Minister Minoru Kihara said 1,000 military personnel were preparing to go to the region, while 8,500 others were on standby. Around 20 military aircraft were dispatched to survey the damage.
Monday's quake shook apartments in the capital Tokyo about 300 kilometres away, where a public New Year greeting event that was to be attended by Emperor Naruhito and his family members was cancelled.
Several major highways were closed around the epicentre, Japan's road operator said, and bullet train services from Tokyo were also suspended.
Japan experiences hundreds of earthquakes every year and the vast majority cause no damage.
The number of earthquakes in the Noto Peninsula region has been steadily increasing since 2018, a Japanese government report said last year.
Japan is haunted by a massive 9.0-magnitude undersea quake off northeastern Japan in 2011 that triggered a tsunami that left around 18,500 people dead or missing.
It also swamped the Fukushima atomic plant, causing one of the world's worst nuclear disasters.
Japan's nuclear authority said there were no abnormalities reported at the Shika atomic power plant in Ishikawa or at other plants after Monday's quake.
In Washington, US President Joe Biden was briefed on Monday's quake and offered Japan "any necessary assistance" to cope with the aftermath.
French President Emmanuel Macron expressed "solidarity" while Italian Prime Minister Giorgia Meloni offered condolences and assistance.