I am excited to learn about the details for complexity-based rate limits, as there are quite a few design considerations involved. For example, how to configure which (numeric) arguments in a GraphQL query actually impact complexity. The example in the blogpost relies on `first`, which seems to be derived from Relay Connections. But implementations might differ and user other arguments.
We did research on solving the DOS issue using static analysis at IBM (cf. https://arxiv.org/pdf/2009.05632.pdf). Our findings were that static analysis allows to determine (relatively strict) upper bounds on query complexity, which we assessed for two production GraphQL APIs (GitHub and Yelp). However, the static analysis requires some configuration to determine the (maximum) size of lists.
I was later involved in productising said research into an API gateway (called DataPower) offered by IBM. We implemented our GraphQL static analysis in a quite flexible and performant way (all of GraphQL's query validation and the static analysis are implemented in C++). The required configuration for the static analysis can be provided using GraphQL schema directives (cf. https://ibm.github.io/graphql-specs/cost-spec.html). Unfortunately, DataPower is quite inaccessible to the common developer.
I find that persisted queries are a very interesting approach to also solve this issue. They still grant developers the full flexibility to define queries during development, but then require providers to validate and vet queries only once before persisting them (instead of validating and vetting queries on every request). This has huge benefits for runtime performance, of course.
Principle 8 mentions "demand control", including "estimating the cost of a query before performing it". This is very much in line with GraphQL API Management work we are doing at IBM Research. I recently wrote about this: https://www.ibm.com/blogs/research/2019/02/graphql-api-manag...
I wonder about the other proposal for demand control, namely that untrusted users "should only send queries that have been preregistered by the authenticated developer of the app". To me, that seems to somewhat negate a great advantage of GraphQL, namely giving clients more flexibility in deciding (on the fly) what data to fetch/mutate.
There are a few libraries that calculate complexity or cost of a GraphQL query, e.g., [1][2]. We think that differences of our approach are that we extract more diverse characteristics, allowing for a wider array of management policies, and that we explicitly decouple management from the backend implementation. In consequence, our approach works irrespective of the programming language used for the GraphQL backend, prevents malicious requests from even hitting the backend, reduces backend load, and lifts developers from having to integrate cost / complexity calculations themselves.
API Harmony is a research prototype we develop here at IBM Research. Using it, we explore things like mining API specifications, static code analysis of web API requests, or API recommendation. Some of this research shows directly in API Harmony (with sometimes good, sometimes fair, and sometimes rather poor results, admittedly), and others are disseminated as more fundamental research results (like our static analysis of web API requests, which made it to the International Conference on Software Engineering this year).
APIful.io is a blog about our work, within API Harmony and beyond. We use it to expose our results and (hopefully) foster discussions.
One of our goals is to mine information about APIs, rather than relying on user input (hence the "catalog"). We still have a long way to go, though. In Web API land, many things follow a power-law distribution: for a few well-known APIs, a lot of information can be found (https://apiharmony-open.mybluemix.net/apis/instagram). For many other APIs, information is sparse and hard to mine.
Adding Schema.org (or similar) annotations would help. But it may be as hard to convince providers to do this, as it is to convince them to write OpenAPI specifications (or similar).
Shameless plug, I know, but I bet Inferno doesn't come close to my latest framework when it comes to the notorious task of writing a TODO app: https://github.com/ErikWittern/TODOit
Mixing code with documentation is one concern. Another issue is that extracting the information needed to create a Swagger from source code is hard, especially when using dynamic languages (JavaScript, Python). One approach is to use dedicated annotators to relate code with parts of an API spec - but then you have the first problem again...
Cool projects, thanks for pointing them out! So, the scrape-html-to Swagger one would basically allow you to keep Swagger up to date as you change the human-readable API documentation? In that case, someone still has to make sure the API documentation is in sync with the source code, right?
What do you think about automatic generation of client-code in various languages?
With regards to the bugs, were they a result of the implementation and the specification getting out of sync? Would automatic means for creating / maintaining specs be able to help here?
Hi, the author here. I think its a good point you make. However, I believe being based on JSON or YAML, many of the specification formats provide at least some degree of readability. Also, one idea of them is to provide the basis for creating consistent, human-targeted documentations (see the right side of http://editor.swagger.io/). One might say these documentations are not good enough, but that would be a discussion about the quality of the tooling, rather than an argument against formal specifications per se.
I am excited to learn about the details for complexity-based rate limits, as there are quite a few design considerations involved. For example, how to configure which (numeric) arguments in a GraphQL query actually impact complexity. The example in the blogpost relies on `first`, which seems to be derived from Relay Connections. But implementations might differ and user other arguments.
We did research in this area (cf. https://arxiv.org/pdf/2009.05632.pdf) and proposed a specification to embed required configuration in the GraphQL schema (cf. https://ibm.github.io/graphql-specs/cost-spec.html).