Microsoft open sources PostgreSQL extensions to muscle in on NoSQL

Analysis When Microsoft rolled out an open source extension stack for PostgreSQL to handle document-style data, it wasn't just taking aim at MongoDB - the dominant NoSQL player - but also blurring the lines between relational and non-relational databases, according to one expert.

Despite the tech giant having built a multitrillion-dollar valuation largely on proprietary software, its latest bid to back open source systems and compete in the document database market promises a schema-light approach popular with developers.

Late last month, Redmond announced two extensions to PostgreSQL, the open source relational database system with roots reaching back to the 1980s.

It has built pg_documentdb_core, a custom PostgreSQL extension that enables support for Binary JavaScript Object Notation (BSON, a binary-encoded serialization of JSON documents), and pg_documentdb_api, a data layer providing MongoDB-compatible commands for create, read, update and delete (CRUD) operations, queries, and index management. They are set to run on the Azure Cosmos DB PostgreSQL database service.

Microsoft also notes that its extensions can be used with FerretDB, an open source MongoDB alternative. FerretDB claims its 2.0 version delivers up to 20x performance improvements for certain workloads over earlier versions, partly due to backend optimizations.

Andrew Pavlo, associate professor of databaseology at Carnegie Mellon University, told The Register the move was another sign that document databases, classified as part of the NoSQL or non-relational database category, would become a feature of relational systems rather than a category on their own.

"The intellectual distance between document/JSON DBMSes and relational DBMSes is shrinking. At some point, the two system categories will be indistinguishable (at least in terms of the data model). The DocumentDB/FerretDB announcement is further evidence of this," he said.

"All the NoSQL systems are becoming relational, except for [key-value database] Redis. They now expose a relational data model with support for nested data (for example, JSON). They also expose a SQL interface, but they can't get themselves to call it SQL for some reason. Instead, they give their query languages a different name - Cassandra has CQL and Aerospike has AQL - and claim it is 'inspired' by SQL. Even MongoDB added support for SQL in their Atlas service in 2022."

MongoDB helped establish the market for document databases in the early 2010s. In 2018, it introduced a Server Side Public License - which requires any organization offering MongoDB as a service to release the source code of the entire service - and also offers proprietary licenses. Its claim to be a favorite among developers is not without justification as it ranks highly on the Stack Overflow survey and counts Wells Fargo bank, Sega, and L'Oréal among its customers.

It was predictably unimpressed by Microsoft's effort to muscle in on its market.

A spokesperson at MongoDB said: "The rise of MongoDB imitators proves our document model is the industry standard. But bolting an API onto a relational database isn't innovation - it's just kicking the complexity can down the road. These 'modern alternatives' come with a built-in sequel: the inevitable second migration when performance, scale, and flexibility hit a wall. Developers building modern, AI-powered applications don't have time for do-overs. MongoDB is purpose-built to get it right the first time."

FerretDB 1.0 launched in 2023, claiming to be "a truly open source MongoDB alternative, built on PostgreSQL, and released under the Apache 2.0 license."

Speaking to The Register, CEO Peter Farkas said FerretDB wanted to work with Microsoft to achieve the long-term aim of building a "more solid foundation for users to have a MongoDB alternative."

"The goal with this is to work broadly among the providers of MongoDB alternatives and have more cooperation. After a while, they would have one solid foundation for MongoDB alternatives, which everybody could build on," he said.

The reason for building on PostgreSQL was that, like Pavlo, Farkas sees the NoSQL group of databases eventually becoming a feature of relational systems.

"From time to time, specialized databases appear on the market, and these would live independently for a good while, but then going by history, large databases like Oracle or PostgreSQL would start to support features, either at their core or through extensions like what we did with Microsoft," he said.

PostgreSQL began adding support for JSON documents in 2013 and has developed it ever since, but that is not enough to create a rival to MongoDB on its own, Farkas argued.

"With PostgreSQL, working with extensions is much easier than getting something into the core database. Even large enterprises like Microsoft may not be able to steer the PostgreSQL community in that way, but extensions provide a very similar experience to the user. It's not complicated to install a Postgres extension and use it alongside PostgreSQL and for now, the innovation with PostgreSQL happens through extensions," he said.

Meanwhile, MongoDB has introduced new data types and features that could not be supported by the current JSON functionality of PostgreSQL, Farkas said. "It would not be possible to implement a performant MongoDB-compatible experience on top of the JSON support in PostgreSQL, and it was never a goal to become MongoDB-compatible."

Through the partnership with Microsoft, Farkas wants to create a developer-friendly experience for a document database without having to get their hands dirty with PostgreSQL itself. But he also wants to build a common standard for other PostgreSQL database services from alternative cloud vendors such as Google's Cloud SQL for PostgreSQL. To this end, he has called for collaboration among vendors to improve compatibility between alternatives to MongoDB.

Industry analysts are skeptical that Microsoft's partnership with FerretDB signals a larger open source shift. Gartner senior director analyst Aaron Rosenbaum said it was unusual for Redmond to form a market partnership - even one just for PR purposes - as they've done with FerretDB and announced it so publicly.

But that does not mean it is likely to support other open source database projects. "We see this as continuing their work with Postgres," he said. "They have been a significant contributor to PostgreSQL. This has no signs of a broadening trend as they don't have that track record of contributing to other open source DBMSes. While Microsoft is very committed to SQL Server continuing, it's also clear that they are very serious about support for Postgres. This isn't dissimilar to their support for Linux and Windows Server on Azure."

Henry Cook, Gartner director analyst, said that the Microsoft extensions to PostgreSQL would continue to strengthen the open source system's position in the market and as an interface for DBMS services.

However, the tie-up with FerretDB was not likely to seriously damage MongoDB either. "MongoDB has established itself in a firm position within the market championing the NoSQL approach. This will provide more competition but there are already other NoSQL offerings out there. MongoDB will continue to defend its position based on its merits," Cook said. ®