Living APIs and the Case for GraphQL(brandur.org)
brandur.org
Living APIs and the Case for GraphQL
https://brandur.org/graphql
20 comments
1. GraphQL is more powerful and you don't get all that for free.
Take a look at Apollo’s work on merging schemas together for a good example.[1]
Personally, I’m working on a proxy layer that will sit in front of a GraphQL API and act as a sort of business rule engine. It analyses the incoming queries to figure out which sets of rules need to be run, and then actually alters the incoming query to add in any additional data points required before forwarding the request to the backend API. Once the combined result comes back, it executes the appropriate rules and filters the output before sending back to the client.
These sorts of solutions don’t have anything to do with APIs. Heck, you could create a GraphQL schema to act as a classic data access layer, and call it internally from your own code.
2. You end up paying more attention to designing a good GraphQL schema/API because of the type system.
I’m not sure where GraphQL will end up either, but I can see a place for it just about anywhere. (I suppose I’m a bit of a kool-aid drinker.. I wasn’t when I first started using GraphQL, but it has grown on me)
[1] https://www.apollographql.com/docs/graphql-tools/schema-stit... [2] https://apis.guru/graphql-voyager/
I've felt similar from working with GraphQL in Rails, but it's not entirely fair to blame the graphql gem for this because Rails has been an MVC-style RESTful web framework for as long as it's lived. Naturally there'll be a lot more exposed wires as you're building up a server that has different requirements for an application architecture.
It's easier said than done but I just think that there needs to be something for GraphQL that Rails was to REST, and for those options to exist outside of Node-land. I don't know what's out there for that, but I have heard a bit about absinthe: https://hexdocs.pm/absinthe/overview.html.
Having gone back to a project using REST and Redux, I miss working with graphql.
It's easier said than done but I just think that there needs to be something for GraphQL that Rails was to REST, and for those options to exist outside of Node-land. I don't know what's out there for that, but I have heard a bit about absinthe: https://hexdocs.pm/absinthe/overview.html.
Having gone back to a project using REST and Redux, I miss working with graphql.
> * Because of this, the actual GraphQL servers get less attention and there's less resources on building them.
By graphql servers do you mean graphql clients on the server?
I'd think there's not much resources on building them because the basics are so trivial, the composition features are rarely necessary so all you need is the ability to POST a query, and parse the resulting JSON document.
My experience of interacting with the Github v4 API was understanding how to make queries:
And then some code afterwards to check for errors and pull out the bits into "native" objects, but that would be no different in an other API style.
By graphql servers do you mean graphql clients on the server?
I'd think there's not much resources on building them because the basics are so trivial, the composition features are rarely necessary so all you need is the ability to POST a query, and parse the resulting JSON document.
My experience of interacting with the Github v4 API was understanding how to make queries:
requests.post('https://api.github.com/graphql', json={
'query': query_string,
'variables': my_vars,
})
And then some code afterwards to check for errors and pull out the bits into "native" objects, but that would be no different in an other API style.
Seems that oData offers all the benefits of graphql in a standardized way, for restful APIs. With the added benefit that it's supported in many tools like ms-excel, tableau, etc.
GraphQL is good for some things, but often pretty dangerous for many of the use cases it was originally introduced for. Without aggressive validation, it does introduce significant blocking and overloading concerns for irresponsible clients or bad actors. I like having a mix of a highly restricted GraphQL API on the read end and a strictly enforced protocol buffer RPC. This enforces good schemas, and encourages extremely slim requests over the wire where possible. It's one very good solution in your toolbelt, and one that integrates really well with a lot of the frontend frameworks out there today. It's just worth being judicious about.
"Aggressive" validation seems to be a part of most libraries for helping people implement GraphQL, so I don't think this is a real issue. I'm assuming you mean stuff like max nodes, max depth, max page size, etc.
It could also be referencing authentication and authorization controls on fields - I think GraphQL makes it too easy for developers to expose private fields without realizing it. Just last week I found a financial institution exposing bank account numbers and routing numbers (not mine) through their GraphQL endpoint - they still haven't fixed it.
This is just as easy to do with REST, though, and not something unique to GraphQL. Most of the time, the problem has to do with specifying fields to exclude rather than fields to expose, which makes it easy to add a private field to a model and have it automatically, unintentionally exposed by the API.
Wow, someone actually wrote a schema exposing that account number. I've seen that with RESTful endpoints because it's so easy for a developer to serialize an entire object (or object graph) without thinking about what's on it.
Interview with Christian Schwendtner
“GraphQL is good, but it’s not an alternative to ‘real’ REST services“
https://jaxenter.com/graphql-good-no-alternative-rest-servic...
https://jaxenter.com/graphql-good-no-alternative-rest-servic...
That's really nice, but "real REST services" have never really been a thing, because the reality of "real REST services" is, at the end of the day, that they're not very useful or convenient for machines to consume.
REST works in browsers because there's an intelligent agent driving the interaction. For an automated system it's just pointless drudgery and unnecessary extra requests and data. GraphQL basically fixes all the slow bits, by allowing for a single query in which the client specifies exactly what they want, it cuts down drastically on both the number of network round-trips and the size of data retrieved. And at the end of the day, that means it also (by and large) cuts down on server resources, because those extra connections and that extra data to extract & format is not free.
So GraphQL ends up being both more convenient and more efficient than "real REST services", and that's before we even get to the self-documentation from the schema and to graphiql explorers, which are superior to what "real REST services" could (but never did in my experience) (even less so as browsers kept on not supporting anything other than GET and POST) have provided.
REST works in browsers because there's an intelligent agent driving the interaction. For an automated system it's just pointless drudgery and unnecessary extra requests and data. GraphQL basically fixes all the slow bits, by allowing for a single query in which the client specifies exactly what they want, it cuts down drastically on both the number of network round-trips and the size of data retrieved. And at the end of the day, that means it also (by and large) cuts down on server resources, because those extra connections and that extra data to extract & format is not free.
So GraphQL ends up being both more convenient and more efficient than "real REST services", and that's before we even get to the self-documentation from the schema and to graphiql explorers, which are superior to what "real REST services" could (but never did in my experience) (even less so as browsers kept on not supporting anything other than GET and POST) have provided.
> because the reality of "real REST services" is, at the end of the day, that they're not very useful or convenient for machines to consume.
Browsers seem to work fine, unless it's just been a tiny human inside my computer this entire time?
Browsers seem to work fine, unless it's just been a tiny human inside my computer this entire time?
Did your browser somehow post this comment on its own, on your behalf, without you being involved in any way?
I'm guessing no, and that's exactly why REST work. The tiny human is not inside the computer, it's sitting between the keyboard and screen, able to (to a very variable extent) intelligently interpret every document's information and react on the fly depending on their whims and needs.
I'm guessing no, and that's exactly why REST work. The tiny human is not inside the computer, it's sitting between the keyboard and screen, able to (to a very variable extent) intelligently interpret every document's information and react on the fly depending on their whims and needs.
> Did your browser somehow post this comment on its own, on your behalf, without you being involved in any way?
No, but that's not what rest means?
It did accurately figure out that a file was CSS and to pain the DOM, JS is a program and to run it, to automatically display any images of the correct type and not explode when it sees a image of a type it doesn't know about, and it showed me actions I could click.
No, but that's not what rest means?
It did accurately figure out that a file was CSS and to pain the DOM, JS is a program and to run it, to automatically display any images of the correct type and not explode when it sees a image of a type it doesn't know about, and it showed me actions I could click.
> No, but that's not what rest means?
It kinda is, if there's no interaction back and forth the entire thing is completely trivial.
> It did accurately figure out that a file was CSS and to pain the DOM, JS is a program and to run it, to automatically display any images of the correct type and not explode when it sees a image of a type it doesn't know about,
Little of which is of any use to two automated systems interacting without human intervention, and the bits which are of use are not hard.
> it showed me actions I could click.
That's the useful bit when working with an API, knowing that an action can be performed and being able to perform that action.
And that's where REST falls down, because at the end of the day it's not very helpful or convenient at programmatically exposing and allowing these interactions for automated systems.
Again, very good when there's an intelligent and flexible meat bag at one end of the interaction, but that's not what APIs are for in colloquial use.
It kinda is, if there's no interaction back and forth the entire thing is completely trivial.
> It did accurately figure out that a file was CSS and to pain the DOM, JS is a program and to run it, to automatically display any images of the correct type and not explode when it sees a image of a type it doesn't know about,
Little of which is of any use to two automated systems interacting without human intervention, and the bits which are of use are not hard.
> it showed me actions I could click.
That's the useful bit when working with an API, knowing that an action can be performed and being able to perform that action.
And that's where REST falls down, because at the end of the day it's not very helpful or convenient at programmatically exposing and allowing these interactions for automated systems.
Again, very good when there's an intelligent and flexible meat bag at one end of the interaction, but that's not what APIs are for in colloquial use.
I wouldn't say GraphQL is an alternative to REST.
GraphQL is (a DSL syntax for) a restricted subset of SQL—one that's restricted enough that it's okay to allow arbitrary clients from the public Internet to request of you.
Like SQL, though, its proper place is "inside" the API abstraction-layer, not "outside" it. GraphQL queries, like SQL queries, are things your backend app-layer logic can generate and send to your DB layer (presuming your DB layer supports GraphQL.)
There are certain APIs where you might want to expose the ability to use GraphQL to modify the composition of the tree of data-structures returned by a particular controller-route. But this doesn't mean you should be using GraphQL "as" your API. GraphQL here is just a parameterization of your API, like result-set pagination. The API itself still needs to exist regardless of GraphQL, and needs to speak something that allows for all the standard CRUD operations.
GraphQL is (a DSL syntax for) a restricted subset of SQL—one that's restricted enough that it's okay to allow arbitrary clients from the public Internet to request of you.
Like SQL, though, its proper place is "inside" the API abstraction-layer, not "outside" it. GraphQL queries, like SQL queries, are things your backend app-layer logic can generate and send to your DB layer (presuming your DB layer supports GraphQL.)
There are certain APIs where you might want to expose the ability to use GraphQL to modify the composition of the tree of data-structures returned by a particular controller-route. But this doesn't mean you should be using GraphQL "as" your API. GraphQL here is just a parameterization of your API, like result-set pagination. The API itself still needs to exist regardless of GraphQL, and needs to speak something that allows for all the standard CRUD operations.
Respectfully, this is false. First, GraphQL is not a subset of SQL, nor is it in any way related to SQL. Second, GraphQL is designed for safe client-server interaction. GraphQL is not a query language in a traditional sense, it is simply a syntax for a consumer to describe how the data it receives should be structured. This does not preclude server-side usage as well, however.
While a real-life application may want to expose a REST interface, there is no inherent reason why a service cannot expose a GraphQL-only API, and each client only consume that API.
While a real-life application may want to expose a REST interface, there is no inherent reason why a service cannot expose a GraphQL-only API, and each client only consume that API.
> First, GraphQL is not a subset of SQL, nor is it in any way related to SQL.
I didn't mean to imply the language syntaxes are related. But they're both syntaxes for relational algebra, and GraphQL has a strict subset of SQL's capabilities in terms of what relational-algebra queries/mutations it can express. (I.e. GraphQL can express joins, but only inner joins with an aggregate projection on the left side; and GraphQL can express projections, but only trivial projections applying the identity function to a subset of the available tuple fields.)
(Note that other, more powerful graph manipulation languages, e.g. Datalog or SPARQL, are also syntaxes for relational algebra. They're similarly limited in some ways compared to SQL—and yet the types of queries they express easily would balloon out to hundreds of lines of SQL.)
> there is no inherent reason why a service cannot expose a GraphQL-only API, and each client only consume that API.
There is no inherent reason why a service cannot expose an SQL API, either, provided you lock down the restrictions on the role the DB is acting as (both in terms of ACLs on DML query types, and in terms of caps on compute time, memory, etc.)
But people don't do that, and they avoid it for more than just the challenging ops story it implies. You don't expose SQL-based APIs to the public internet, because you tend to build APIs to service the needs of particular clients with particular use-cases, and those use-cases can be optimized for by adding things like caching middleware, but only if you can filter out and tag the particular use-case each request is for. REST lets you do that; the use-cases are directly named by the (method + origin + path), and the semantics are already there, built into the protocol, for caching them, substituting them, access-controlling them, etc.
GraphQL, like SQL, doesn't have any place for naming the use-case of a request, in the request. So, if you want any of the useful properties like cacheability or access-control or whatever else to apply, you probably want your GraphQL request to appear "inside" a RESTful request, such that you still have individual REST endpoints for particular use-cases.
I didn't mean to imply the language syntaxes are related. But they're both syntaxes for relational algebra, and GraphQL has a strict subset of SQL's capabilities in terms of what relational-algebra queries/mutations it can express. (I.e. GraphQL can express joins, but only inner joins with an aggregate projection on the left side; and GraphQL can express projections, but only trivial projections applying the identity function to a subset of the available tuple fields.)
(Note that other, more powerful graph manipulation languages, e.g. Datalog or SPARQL, are also syntaxes for relational algebra. They're similarly limited in some ways compared to SQL—and yet the types of queries they express easily would balloon out to hundreds of lines of SQL.)
> there is no inherent reason why a service cannot expose a GraphQL-only API, and each client only consume that API.
There is no inherent reason why a service cannot expose an SQL API, either, provided you lock down the restrictions on the role the DB is acting as (both in terms of ACLs on DML query types, and in terms of caps on compute time, memory, etc.)
But people don't do that, and they avoid it for more than just the challenging ops story it implies. You don't expose SQL-based APIs to the public internet, because you tend to build APIs to service the needs of particular clients with particular use-cases, and those use-cases can be optimized for by adding things like caching middleware, but only if you can filter out and tag the particular use-case each request is for. REST lets you do that; the use-cases are directly named by the (method + origin + path), and the semantics are already there, built into the protocol, for caching them, substituting them, access-controlling them, etc.
GraphQL, like SQL, doesn't have any place for naming the use-case of a request, in the request. So, if you want any of the useful properties like cacheability or access-control or whatever else to apply, you probably want your GraphQL request to appear "inside" a RESTful request, such that you still have individual REST endpoints for particular use-cases.
You wrote..
> GraphQL, like SQL, doesn't have any place for naming the use-case of a request
This is not true, see the docs:
> The operation name is a meaningful and explicit name for your operation. It can be very useful for debugging and server-side logging reasons
https://graphql.org/learn/queries/
You wrote...
> There is no inherent reason why a service cannot expose an SQL API
GraphQL let's you query across backends, such as relational (SQL), graph databases, REST APIs, it is backend agnostic. You're comparing apples to oranges, SQL & GraphQL are different abstractions.
> GraphQL, like SQL, doesn't have any place for naming the use-case of a request
This is not true, see the docs:
> The operation name is a meaningful and explicit name for your operation. It can be very useful for debugging and server-side logging reasons
https://graphql.org/learn/queries/
You wrote...
> There is no inherent reason why a service cannot expose an SQL API
GraphQL let's you query across backends, such as relational (SQL), graph databases, REST APIs, it is backend agnostic. You're comparing apples to oranges, SQL & GraphQL are different abstractions.
SQL is backend agnostic. And I'm constantly surprised that people aren't aware of this, especially the people who think that a DB needs to be "NoSQL" just because it necessarily can't support all the features that SQL offers. No RDBMS supports 100% of the features in the SQL standard, either! It's okay to "support SQL" in a way where you just throw errors if 90% of SQL's syntax is used! Many systems do that! Clients are Required by the SQL standard to cope with that!
Every modern DB labelled "NewSQL" is fundamentally just a "NoSQL" database that has been made to speak SQL. Sometimes it is, in fact, literally a previously-built NoSQL database, with an SQL adapter layer like https://en.wikipedia.org/wiki/Presto_(SQL_query_engine) used as middleware.
And your application can speak SQL, too! It's not that hard to:
1. grab an SQL2016.y grammar file and generate a parser for it in your language-of-choice;
2. expose a port on your daemon that speaks a protocol wire-compatible with some particular RDBMS's wire protocol, e.g. the https://www.postgresql.org/docs/current/static/protocol.html (like CockroachDB has done.)
If you do these two things, then other programs can connect to your app as a database, and use their ORM tools to handle your data model. (Consider this option if you're ever developing e.g. an MMORPG server; it can save hundreds of hours of work in exposing new APIs and deprecating old ones, for the cost of a little front-loaded work.)
---
But I digress.
You can make any application layer, or database layer, speak GraphQL. Because it's a syntax for relational algebra, and relational algebra is just "How You Describe Relationships When Making Requests." It has nothing to do with the fundamental underlying schema of the data; it's a language for communicating your intent—the things you want to get, or the things you want to change—by naming them through their relationships to other things.
Now replace "GraphQL" with "SQL" in the above and notice that nothing changes.
Every modern DB labelled "NewSQL" is fundamentally just a "NoSQL" database that has been made to speak SQL. Sometimes it is, in fact, literally a previously-built NoSQL database, with an SQL adapter layer like https://en.wikipedia.org/wiki/Presto_(SQL_query_engine) used as middleware.
And your application can speak SQL, too! It's not that hard to:
1. grab an SQL2016.y grammar file and generate a parser for it in your language-of-choice;
2. expose a port on your daemon that speaks a protocol wire-compatible with some particular RDBMS's wire protocol, e.g. the https://www.postgresql.org/docs/current/static/protocol.html (like CockroachDB has done.)
If you do these two things, then other programs can connect to your app as a database, and use their ORM tools to handle your data model. (Consider this option if you're ever developing e.g. an MMORPG server; it can save hundreds of hours of work in exposing new APIs and deprecating old ones, for the cost of a little front-loaded work.)
---
But I digress.
You can make any application layer, or database layer, speak GraphQL. Because it's a syntax for relational algebra, and relational algebra is just "How You Describe Relationships When Making Requests." It has nothing to do with the fundamental underlying schema of the data; it's a language for communicating your intent—the things you want to get, or the things you want to change—by naming them through their relationships to other things.
Now replace "GraphQL" with "SQL" in the above and notice that nothing changes.
1 - I don't have to fork my database to use graphQL... it was designed to be an API gateway.
2 - forking databases to speak the same language still doesn't solve the issue of how to coalesce data from multiple backends into one response. You need an API gateway for that.
3 - forking a db would take months or years for me. I added graphQL to my stack in a few hours without modifying my backends, again because it's designed as an API gateway.
Yes they both declare data requirements. But no they aren't both relational and only one of these two technologies was designed as an API gateway.
2 - forking databases to speak the same language still doesn't solve the issue of how to coalesce data from multiple backends into one response. You need an API gateway for that.
3 - forking a db would take months or years for me. I added graphQL to my stack in a few hours without modifying my backends, again because it's designed as an API gateway.
Yes they both declare data requirements. But no they aren't both relational and only one of these two technologies was designed as an API gateway.
[deleted]
You seem to be confusing graphql with something graph-like but completely unrelated. It was fully designed as an external API protocol. At no point is it intended to sit between your server and your DB, though I guess you could use it that way if you really wanted to.
> The API itself still needs to exist regardless of GraphQL
It really, absolutely, emphatically, does not.
> The API itself still needs to exist regardless of GraphQL
It really, absolutely, emphatically, does not.
I think maybe we're using "API" differently, here? GraphQL lets you query and mutate an object graph, but things exist that can't be (canonically, compactly) represented as a tree of objects. And usually at least a few of those things are necessary for any given API.
For example, how do you implement your AAA login logic using GraphQL? You don't, right? Logging in isn't really anything to do with querying or mutating an object tree. (You can deform your logic into making it so, but you'll be breaking things like the ability to do OpenID Connect.)
How do you implement an endpoint to stream chat message events in GraphQL? You don't, right? You want a stream of trees? A tree of stream-poll-results? There's an impedance mismatch here.
You want to use GraphQL to upload attachments? Use GraphQL for webhooks? Use GraphQL for Operational Transformations in a Google Docs-alike? No, no, no.
GraphQL is useful for implementing the message-format of particular endpoints of an API. It doesn't work for all endpoints. Unless your API is entirely just about a particular object graph, you still need to put your GraphQL inside a larger abstraction that distinguishes the GraphQL endpoints from the other endpoints.
For example, how do you implement your AAA login logic using GraphQL? You don't, right? Logging in isn't really anything to do with querying or mutating an object tree. (You can deform your logic into making it so, but you'll be breaking things like the ability to do OpenID Connect.)
How do you implement an endpoint to stream chat message events in GraphQL? You don't, right? You want a stream of trees? A tree of stream-poll-results? There's an impedance mismatch here.
You want to use GraphQL to upload attachments? Use GraphQL for webhooks? Use GraphQL for Operational Transformations in a Google Docs-alike? No, no, no.
GraphQL is useful for implementing the message-format of particular endpoints of an API. It doesn't work for all endpoints. Unless your API is entirely just about a particular object graph, you still need to put your GraphQL inside a larger abstraction that distinguishes the GraphQL endpoints from the other endpoints.
I share most of your points, however I think there are two that actually play well with GraphQL.
> You want a stream of trees?
I was under the impression that is what GraphQL subscriptions are for but I haven't had the chance to try them. Isn't that so?
> You want to use GraphQL to upload attachments?
I have been using apollo-upload and it works quite well.
> You want a stream of trees?
I was under the impression that is what GraphQL subscriptions are for but I haven't had the chance to try them. Isn't that so?
> You want to use GraphQL to upload attachments?
I have been using apollo-upload and it works quite well.
I maintain a relatively big REST API. There are hundreds of different entities with very deep relationships.
Our solution is exactly like Stripe's. Clients ask for the relationships they want to fetch using the query string.
It's not ideal. Seems like a hack. But it works.
On the bright side, I can query the system using curl.
What should be my incentive to consider switching to GraphQL for this project?
What should be my incentive to consider GraphQL for my next project?
Our solution is exactly like Stripe's. Clients ask for the relationships they want to fetch using the query string.
It's not ideal. Seems like a hack. But it works.
On the bright side, I can query the system using curl.
What should be my incentive to consider switching to GraphQL for this project?
What should be my incentive to consider GraphQL for my next project?
I think the big (only?) advantage that GraphQL has in this case is you get "API documentation" out of the box as a frontend developer.
The tradeoff is that I need to use GraphiQL or similar tooling to leverage this auto-complete, type-safe magic and that using curl becomes more painful.
I'm curious: Suppose you converted your REST endpoints into GraphQL endpoints in the simplest way possible by changing the query-params to root GraphQL nodes. So there's no "graph" really. Just lots of top level nodes. This reduces the API docs overhead and nothing else.
Would you switch? Is your existing tooling to extract API docs already good enough that this isn't a problem?
The tradeoff is that I need to use GraphiQL or similar tooling to leverage this auto-complete, type-safe magic and that using curl becomes more painful.
I'm curious: Suppose you converted your REST endpoints into GraphQL endpoints in the simplest way possible by changing the query-params to root GraphQL nodes. So there's no "graph" really. Just lots of top level nodes. This reduces the API docs overhead and nothing else.
Would you switch? Is your existing tooling to extract API docs already good enough that this isn't a problem?
https://docs.rechat.com
This is our API Documentation and it's generated automatically from our E2E tests. It's almost free for us to maintain.
Can you tell me if this is OK enough comparing to a GraphQL that wouldn't justify a rewrite?
This is our API Documentation and it's generated automatically from our E2E tests. It's almost free for us to maintain.
Can you tell me if this is OK enough comparing to a GraphQL that wouldn't justify a rewrite?
Do you consume the API as well?
I'm currently writing the GraphQL API to replace our REST API. It is a lot of work, but when I have to put on my "front-end" hat it will be much easier to work with. I think the biggest payoff is for consumers.
I'm currently writing the GraphQL API to replace our REST API. It is a lot of work, but when I have to put on my "front-end" hat it will be much easier to work with. I think the biggest payoff is for consumers.
I think a good reason is developers/users don’t have to learn the verbiage/tooling that each vendor uses, especially for more complex rest APIs that offer features like the fields query param.
Just FYI, you can query GraphQL with cURL, too. It's just HTTP, it doesn't require any fancy special clients. `fetch` in the browser works fine for talking to GraphQL as well.
That’s literally true, but on balance it’s way easier to use curl for GET requests to a REST API than it is to use curl for GraphQL queries. At least if we’re talking about “let me curl that route real quick to see what the data looks like”.
You could have a REST API that requires 100s of query params to get at some simple data, or a graphQL API which exposes complex data serialized into a single field. Both can be as easy or as hard to query as you want them to be.
The standard arguments for GraphQL all apply here. It:
- Lets you send only the information asked for. For example, you may be sending 50 fields when the client is only interested in 2.
- Lets you evolve the API more safely
- Enables better tooling for people who need to explore your API, like GraphiQL
- Lets you send only the information asked for. For example, you may be sending 50 fields when the client is only interested in 2.
- Lets you evolve the API more safely
- Enables better tooling for people who need to explore your API, like GraphiQL
One critique I haven't seen about GraphQL is that it's less human-readable. I know that "GraphQL is unapologetically driven by the requirements of views and the front‐end engineers that write them"[0], but if everything switched to GraphQL endpoints, I'd miss having URLs you could change by hand :/
[0]: https://facebook.github.io/graphql/October2016/#sec-Overview
[0]: https://facebook.github.io/graphql/October2016/#sec-Overview
> I'd miss having URLs you could change by hand :/
URLs you can change by hand in API calls? When I'm at that level, I can just as easily edit the call body.
> One critique I haven't seen about GraphQL is that it's less human-readable.
My experience is the opposite, reading a complete graphql query is way easier than reading a mix of URL, query parameters and entity body, even more so when I need multiple such queries to replace a single graphql query.
And the query tells me exactly what's in the result, to boot, so it acts as documentation for the data-extraction code.
URLs you can change by hand in API calls? When I'm at that level, I can just as easily edit the call body.
> One critique I haven't seen about GraphQL is that it's less human-readable.
My experience is the opposite, reading a complete graphql query is way easier than reading a mix of URL, query parameters and entity body, even more so when I need multiple such queries to replace a single graphql query.
And the query tells me exactly what's in the result, to boot, so it acts as documentation for the data-extraction code.
> URLs you can change by hand in API calls?
No, that's not what I meant. I meant that RESTful APIs relate the URL to the data in a way that isn't true in GraphQL APIs.
Take reddit, for instance. I can clearly see a username in the url: https://www.reddit.com/user/masklinn
Swapping one out behaves as I'd expect: https://www.reddit.com/user/faitswulff
And tacking on a format type to the end exposes the content in the manner I'd expect: https://www.reddit.com/user/masklinn.json
The conflation of route and content is frustrating for developing single page apps, but it maps better to my understanding of disk files and folders, thus making the URL more human readable and easier to toggle and change by hand.
If everything was an SPA, you'd have to interact with the data through the frontend, which speaks to some singular GraphQL endpoint, hence my mentioning that GraphQL itself mentions that it is "unapologetic" about its front-end orientation.
No, that's not what I meant. I meant that RESTful APIs relate the URL to the data in a way that isn't true in GraphQL APIs.
Take reddit, for instance. I can clearly see a username in the url: https://www.reddit.com/user/masklinn
Swapping one out behaves as I'd expect: https://www.reddit.com/user/faitswulff
And tacking on a format type to the end exposes the content in the manner I'd expect: https://www.reddit.com/user/masklinn.json
The conflation of route and content is frustrating for developing single page apps, but it maps better to my understanding of disk files and folders, thus making the URL more human readable and easier to toggle and change by hand.
If everything was an SPA, you'd have to interact with the data through the frontend, which speaks to some singular GraphQL endpoint, hence my mentioning that GraphQL itself mentions that it is "unapologetic" about its front-end orientation.
> No, that's not what I meant. I meant that RESTful APIs relate the URL to the data in a way that isn't true in GraphQL APIs.
So 1. you're talking about something completely unrelated to TFA and 2. you're talking nonsense.
> Take reddit, for instance. I can clearly see a username in the url: https://www.reddit.com/user/masklinn
That's a UI component, and an explicit decision of the system's designers.
It, again, has nothing do do with APIs.
Hell, it doesn't even have anything to do with RESTful systems, https://www.reddit.com/lfskfsh8432kdb60 would be just as RESTful.
> The conflation of route and content is frustrating for developing single page apps, but it maps better to my understanding of disk files and folders, thus making the URL more human readable and easier to toggle and change by hand.
Cool, so use that for situations where direct human interactions is expected or convenient, what's the issue?
> If everything was an SPA, you'd have to interact with the data through the frontend
Which you already do, the URL is part of the frontend.
And using SPAs does not preclude providing nice (and user-friendly) URLs e.g. with https://developer.mozilla.org/en-US/docs/Web/API/History
So 1. you're talking about something completely unrelated to TFA and 2. you're talking nonsense.
> Take reddit, for instance. I can clearly see a username in the url: https://www.reddit.com/user/masklinn
That's a UI component, and an explicit decision of the system's designers.
It, again, has nothing do do with APIs.
Hell, it doesn't even have anything to do with RESTful systems, https://www.reddit.com/lfskfsh8432kdb60 would be just as RESTful.
> The conflation of route and content is frustrating for developing single page apps, but it maps better to my understanding of disk files and folders, thus making the URL more human readable and easier to toggle and change by hand.
Cool, so use that for situations where direct human interactions is expected or convenient, what's the issue?
> If everything was an SPA, you'd have to interact with the data through the frontend
Which you already do, the URL is part of the frontend.
And using SPAs does not preclude providing nice (and user-friendly) URLs e.g. with https://developer.mozilla.org/en-US/docs/Web/API/History
Thanks for this, I've totally changed my mind!
Did you think about masklinn's comment for 5 minutes before replying?
I can only assume that the people downvoting this comment didn't see the comment I replied to before it was edited. The "5 minutes" thing was a direct reference.
Agreed. I love being able to say "this is the data I want and the shape I want it in" and being able to get exactly that back.
GraphQL has seen a lot of adoption, but the majority will be private APIs rather than public APIs like Github's.
There's no rules about how you have to use it, but the problem it was built to solve was communication between first-party applications and backends.
There's no rules about how you have to use it, but the problem it was built to solve was communication between first-party applications and backends.
I think GraphQL found it's place between backend and frontend as a data layer. Airbnb just wrote an article detailing this very use-case [0]. Perhaps it isn't suitable for public APIs.
[0]: https://medium.com/airbnb-engineering/reconciling-graphql-an...
[0]: https://medium.com/airbnb-engineering/reconciling-graphql-an...
Good "state of the state" esque writeup. Thanks, Brandur.
I've spent the past three years using Django Rest Framework. I switched over to using a GraphQL (Graphene as the provider, Apollo as the client) and it's been fantastic thus far. Easy to pick up, great documentation. Will learn more as we grow.
I've spent the past three years using Django Rest Framework. I switched over to using a GraphQL (Graphene as the provider, Apollo as the client) and it's been fantastic thus far. Easy to pick up, great documentation. Will learn more as we grow.
I'm not sure I agree with "great documentation". It's a pet peeve of mine, but I don't like it when the tutorial for a server side program uses ES6 modules and decorators. This means I need to set up a build system for my back end. Which is just annoying. I can forgive ES6 modules, since they're experimental in Node 10, but decorators are still on stage 2. Don't write your tutorials with experimental features.
Seconded, I look forward to each new article you post (Brandur).
One very interesting project is rejoiner: https://github.com/google/rejoiner
It translates GraphQL calls to gRPC queries.
It translates GraphQL calls to gRPC queries.
> You should find yourself being able to build a query that delves 4+ relations deep without much trouble.
How is that a good thing from the server’s point of view?
As mentioned elsewhere, almost no one talks avout the server shen talking about GraphQL.
When you can construct an ad-hoc query 4+ relations deep on the client, how can anyone efficiently handle this on the server? You can’t even properly cache GraphQL requests.
How is that a good thing from the server’s point of view?
As mentioned elsewhere, almost no one talks avout the server shen talking about GraphQL.
When you can construct an ad-hoc query 4+ relations deep on the client, how can anyone efficiently handle this on the server? You can’t even properly cache GraphQL requests.
This is the problem I had with graphql. Indeed it simplifies the interface between the backend and the frontend of your application (that is when the api can be cleanly mapped to a graph) but it doesn't really solve the problem of minimizing roundtrips, it just moves it between the backend and the database. Because the database I was querying was not on the same network I was losing seconds (!) doing roundtrips on the network gathering all the data necessary for a single complex graphql query. You may say: well use something like postgraphile but what if you did not designed the database and so it doesn't map cleanly to your graphql object tree? I had to waste a week implementing a postgres query generator so I can remove the latency. Not fun, especially when you have deadlines.
Went with graphql where I'm at, but ended up mostly embedding another query language on a method that most querying happens through. The DB is an entity component system, so Entity table has an id column & the rest of the tables are keyed off that, makes it that any foreign key can reference any entity. So the query language is a JSON blob of { id: "asdf", allOf: { ComponentName: { field: "hasvalue" } }, optional: { OtherComponent: { reffield: { optional: { AnotherComponent: {} } } } } } where it's going off of viewing entities as a bag of components
Initially we were just exposing a big Entity type in GraphQL which had a field for each component. Didn't feel like there was enough control over caching & prefetching. Current system is kind of nice because one reads/writes to redux, calls commit on ids to submit to server, querying just populates the redux store
Initially we were just exposing a big Entity type in GraphQL which had a field for each component. Didn't feel like there was enough control over caching & prefetching. Current system is kind of nice because one reads/writes to redux, calls commit on ids to submit to server, querying just populates the redux store
That blog is just beautiful. I really like this simple style which still looks really well polished.
I definitely agree. Here's the source if you're interested: https://github.com/brandur/sorg
coincidentally, the site for GRPC is similar in design, with a small text sidebar with anchor links:
https://grpc.io/docs/guides/
https://grpc.io/docs/guides/
Great write-up. One aspect of GraphQL that I've found difficult is communicating errors to consumers in a developer-friendly way. What is the best way for GraphQL server to communicate to the client the equivalent of HTTP status codes 500, 404, etc?
The approach I recently implemented was to add a "type" or "code" property with standardized values to error objects in the GraphQL response (similar to error codes in Stripes REST API [0]). The client can then check those "errors" objects for specific codes and behave accordingly.
[0] https://stripe.com/docs/error-codes
The approach I recently implemented was to add a "type" or "code" property with standardized values to error objects in the GraphQL response (similar to error codes in Stripes REST API [0]). The client can then check those "errors" objects for specific codes and behave accordingly.
[0] https://stripe.com/docs/error-codes
This is in the docs.
https://graphql.org/learn/serving-over-http/
> [a] query might result in some data and some errors, and those should be returned in a JSON object of the form {data: {}, errors: [] }
Also checkout https://github.com/kadirahq/graphql-errors which exposes a standard `path` field for each error, so the client has a standard way to know which part of the query resulted in an error.
If a mutation fails, for example a bank withdrawal fails because of insufficient funds, that is not necessarily an "error". To quote "code complete":
> Exceptions should be reserved for what's truly exceptional.
I'd send a mutation like this:
In the case you shove arbitrary error codes in strings, you lose the benefits graphQL brings, like tracking usage of a field, self documenting & introspection, solving for over/under fetching, and the ability to revise these codes without versioning your mutations.
https://graphql.org/learn/serving-over-http/
> [a] query might result in some data and some errors, and those should be returned in a JSON object of the form {data: {}, errors: [] }
Also checkout https://github.com/kadirahq/graphql-errors which exposes a standard `path` field for each error, so the client has a standard way to know which part of the query resulted in an error.
If a mutation fails, for example a bank withdrawal fails because of insufficient funds, that is not necessarily an "error". To quote "code complete":
> Exceptions should be reserved for what's truly exceptional.
I'd send a mutation like this:
withdrawFunds($amount: 100, $currency: "USD") {
didSucceed
didOverdraft
canRetry
}
In the case you shove arbitrary error codes in strings, you lose the benefits graphQL brings, like tracking usage of a field, self documenting & introspection, solving for over/under fetching, and the ability to revise these codes without versioning your mutations.
Those were actually very good points, specially the interaction between client / server being better understood due to the queries being made per purpose and not just trying to obey one of many REST conventions that the server provides.
graphql feels like I'm going back to the days of SOAP/RPC calls, maybe I'm just getting old.
GraphQL is so vastly superior for validation, filtering, nested queries, relations, documentation, discoverability...
You can even have a mutation save a nested object (fragmenting it and passing it down to other handlers), which is so great.
You can make a GraphQL endpoint quite fast with https://github.com/graphql/graphql-js
You can even have a mutation save a nested object (fragmenting it and passing it down to other handlers), which is so great.
You can make a GraphQL endpoint quite fast with https://github.com/graphql/graphql-js
I’ve been doing work with GraphQL for about 1.5 years now. It took a while to wrap my head around but now it’s amazing to work with. I’ve been working with dynamic schema generation, multi-tenant, completely dynamic gql servers, building dynamic gql queries on the client. It’s enabled some pretty amazing functionality that would be more difficult to reproduce with RESTful technologies.
What you described seems very interesting, is there some place to see your work or any writeups to read?
Most of the backend engineers I worked with don't really get or see advantages on why GraphQl is good. Also it's not easy to build GraphQL API on server side compared to REST. There are tons of evolved libraries for REST.
I personally think that transition from REST will be incremental and via JSON route.
As someone doing full stack development, I also don't see any benift, specially now that REST tooling is finally catching up with SOAP and I don't do hype driven development.
What are the benefits of using GraphQL visavis grpc?
grpc is a bog-standard "shallow" RPC: client calls an endpoint, client gets a result, if client needs sub-results it calls more endpoints.
GraphQL queries have depth, when you call a toplevel endpoint which returns a non-trivial type you can select fields of the sub-resources to fetch, in fact you can also select just a slice of the top-level resource as well.
Let's say HN had a grpc API and you wanted the top comment of each article of the main page, unless it provided a very specific endpoint doing that you'd probably have to list the front page (and get a whole host of crap you don't care for e.g. the title, the destination URL, the score, the author, …), and for each article get its comments (again with all sorts of metadata you may not care for). In GraphQL, you'd select all but exactly what you need at once.
GraphQL queries have depth, when you call a toplevel endpoint which returns a non-trivial type you can select fields of the sub-resources to fetch, in fact you can also select just a slice of the top-level resource as well.
Let's say HN had a grpc API and you wanted the top comment of each article of the main page, unless it provided a very specific endpoint doing that you'd probably have to list the front page (and get a whole host of crap you don't care for e.g. the title, the destination URL, the score, the author, …), and for each article get its comments (again with all sorts of metadata you may not care for). In GraphQL, you'd select all but exactly what you need at once.
How does the backend ensure complex queries are efficient and that the client can't make a costly query without realizing it?
graphql does not provide for more complex queries than a regular RPC, the API designers decides specifically what they want to provide at each level.
What they do is coalesce all those separate queries into one, and actually allow for sub-query optimisations if you feel like it.
So the answer is that it does not, any more than e.g. grpc does not.
What they do is coalesce all those separate queries into one, and actually allow for sub-query optimisations if you feel like it.
So the answer is that it does not, any more than e.g. grpc does not.
Good question!
I'm guessing typical implementations would involve a lot of caching.
It would be nice to hear about people's experiences though.
I'm guessing typical implementations would involve a lot of caching.
It would be nice to hear about people's experiences though.
gRPC makes it easy to talk to a server with flat input/output types like in a function call. GraphQL is more a query language.
In some ways, analogous to your client using SQL instead of an ORM to query a system.
In some ways, analogous to your client using SQL instead of an ORM to query a system.
No one in their right mind would ever choose to implement GraphQL rather than a REST api. The complexity difference is just too great. Maybe have your primary API as REST then as some kind of janky caching thing use GraphQL.
The only experience I have had calling GraphQL came from github and it was not great. The API was responsive but when I was trying to figure out syntax I was longing for a much clearer REST end point that would give me the data I was after.
The only experience I have had calling GraphQL came from github and it was not great. The API was responsive but when I was trying to figure out syntax I was longing for a much clearer REST end point that would give me the data I was after.
Okay, I'll bite. Rather than simply shitting on a technology that thousands of developers actually like and have used to positive effect, and stating that none of those thousands of developers are "in their right mind", could you share some of your specific experiences and insights so people can understand why you hold the position you're sharing?
What experiences have you had implementing GraphQL APIs, and what specific complexity differences did you observe?
What experiences have you had in maintaining living REST APIs used by evolving frontend clients, while balancing backwards compatibility concerns for supporting older clients? How has this compared to doing the same with a GraphQL API?
What experiences have you had in developing front-end web and/or mobile applications that consume a GraphQL API? How has that compared to consuming a REST API?
Have you invested any time in learning the GraphQL syntax and understanding the goals and intentions of the project, or did you just use it that one time with GitHub's API and get a bad taste because it was unfamiliar?
I will admit that for APIs provided for 3rd-party public consumption (like GitHub's), GraphQL is not as much of a natural fit for consumers (where, as long as GraphQL is not widely understood, it may just be a "figure out syntax" obstacle) as it can be for in-house APIs consumed by 1st-party front-end applications that utilize tooling built around GraphQL as an advantage. Nonetheless, I would be hesitant to advocate for REST over GraphQL in all cases (as you have) just because I had a bad experience consuming an API that used GraphQL when I was unfamiliar with it.
I'm a backend engineer and I maintain a living GraphQL API consumed by web and mobile engineers on my team. I also helped in our migration from REST to GraphQL, and it's been an overall positive experience for me and the rest of my team. If you stand by your statement that I am out of my mind, I hope you have some relevant experience that you can share to back it up. I doubt you do - your comment comes across more as vague new-paradigm-hate than as experience-based wisdom - but I would be happy to be wrong.
What experiences have you had implementing GraphQL APIs, and what specific complexity differences did you observe?
What experiences have you had in maintaining living REST APIs used by evolving frontend clients, while balancing backwards compatibility concerns for supporting older clients? How has this compared to doing the same with a GraphQL API?
What experiences have you had in developing front-end web and/or mobile applications that consume a GraphQL API? How has that compared to consuming a REST API?
Have you invested any time in learning the GraphQL syntax and understanding the goals and intentions of the project, or did you just use it that one time with GitHub's API and get a bad taste because it was unfamiliar?
I will admit that for APIs provided for 3rd-party public consumption (like GitHub's), GraphQL is not as much of a natural fit for consumers (where, as long as GraphQL is not widely understood, it may just be a "figure out syntax" obstacle) as it can be for in-house APIs consumed by 1st-party front-end applications that utilize tooling built around GraphQL as an advantage. Nonetheless, I would be hesitant to advocate for REST over GraphQL in all cases (as you have) just because I had a bad experience consuming an API that used GraphQL when I was unfamiliar with it.
I'm a backend engineer and I maintain a living GraphQL API consumed by web and mobile engineers on my team. I also helped in our migration from REST to GraphQL, and it's been an overall positive experience for me and the rest of my team. If you stand by your statement that I am out of my mind, I hope you have some relevant experience that you can share to back it up. I doubt you do - your comment comes across more as vague new-paradigm-hate than as experience-based wisdom - but I would be happy to be wrong.
I use it everyday and love it. I write both sides of it and it's paradise compared to working with rest. I'd pick my stack again any day: typescript, react, react-apollo, graphql-tools, apollo-express. I'm able to move so damn fast, it's a breath of fresh air.
Curious about your stack - what's your backend in and what GraphQL provider do you use?
* GraphQL definitely seems to be more popular in the context of the client. A lot of the conversation is around client-side libraries like Apollo and Relay and how to consume APIs.
* Because of this, the actual GraphQL servers get less attention and there's less resources on building them.
* GraphQL on the client is a huge win in my opinion and I've never really heard anyone who's used it disagree with that.
* BUT, GraphQL on the server is absolutely much harder to write than a REST API. I've found two main reasons for this though:
Yes you can absolutely design great RESTful APIs with many of the same features as GraphQL. Even at Shopify we offer a `fields` query param in REST to only select a subset of fields. Guess what? Barely anyone uses this which shouldn't be surprising.
JSON API and companies like Stripe with their `expand` feature offer equivalents to GraphQL but it's non-standard and way harder to do since you're inventing it each time. There's also no standard universal clients for these.
Greenspun's rule applies here in my opinion (slightly exaggerated):
> Any sufficiently complicated RESTful API contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of GraphQL.
I don't know where this will all end up. I do know that GraphQL isn't perfect and it's still early. Public APIs are tough right now because developers are so used to RESTful APIs, but at least at Shopify, we believe that will shift over time.