A Virgin Orbit Boeing 747 carrying the 70-foot (21-metre) rocket took off from a spaceport in Cornwall, southwest England, at 2202 GMT.
The rocket then detached from the aircraft and ignited as planned at a height of 35,000 feet over the Atlantic Ocean to the south of Ireland at around 2315 GMT.
But in a series of tweets as the rocket was due to enter orbit and discharge its nine satellites, Virgin Orbit said: "We appear to have an anomaly that has prevented us from reaching orbit. We are evaluating the information.
"As we find out more, we're removing our previous tweet about reaching orbit. We'll share more info when we can."
The aircraft returned as planned to a subdued Spaceport Cornwall, a consortium that includes Virgin Orbit and the UK Space Agency, at Cornwall Airport Newquay.
The launch was the first from UK soil. UK-produced satellites have previously had to be sent into orbit via foreign spaceports.
Had the mission been successful, the UK would have been one of only nine countries that could launch craft into Earth's orbit.
"Joining that really exclusive club of launch nations is so important because it gives us our own access to space... that we've never had before here in the UK," Spaceport Cornwall chief Melissa Thorpe told BBC television before the launch.
Hundreds of people watched the launch, named "Start Me Up" after the Rolling Stones song.
The satellites were to have a variety of civil and defence functions, from sea monitoring to help countries detect people smugglers to space weather observation.
The number of space bases in Europe has grown in recent years due to the commercialisation of space.
For a long time, satellites were primarily used for institutional missions by national space agencies but most of Europe's spaceport projects are now private-sector initiatives.
The market has exploded with the emergence of small start-ups, modern technology making both rockets and satellites smaller, and the rapidly growing number of applications for satellites.
Some 18,500 small satellites -- those weighing less than 500 kilograms (1,100 pounds) -- are expected to be launched between 2022 and 2031, compared with 4,600 in the previous decade.
Campaigners, however, criticised the launch.
"Space is the new frontier for military escalation and spending with no real public scrutiny or accountability," said Campaign for Nuclear Disarmament (CND) general secretary Kate Hudson.
Drone Wars director Chris Cole denounced a "space arms race which will inevitably lead to greater risk of instability and conflict".