Qb: Database toolkit for Go(qb.readme.io)
qb.readme.io
Qb: Database toolkit for Go
https://qb.readme.io
29 comments
I find by far the most useful aspect of ORMs is the query builder, when done well. Much of the time I'm telling my ORM to output in core data types for speed anyway. Having a query builder that allows me to create my own methods for encapsulating common query actions is too useful to give up though.
For example: my @movies = Schema->Movie->recent->starring("brad pitt")->all;, where recent is a method to restrict movies by age, and starring is a method to join on the actors table and restrict to a specific actor.
For example: my @movies = Schema->Movie->recent->starring("brad pitt")->all;, where recent is a method to restrict movies by age, and starring is a method to join on the actors table and restrict to a specific actor.
I've only used SQLAlchemy, but I was seriously disappointed in its query builder interface. Building SQL queries seems like something that should be straightforward, but SQLAlchemy seems to throw Python magic at every thing, making the API very difficult to reason about. Worse, the documentation is very difficult to locate and navigate through. :(
I suspect this is particular to SQLAlchemy, but if not I'd prefer just to roll my own SQL going forward.
I suspect this is particular to SQLAlchemy, but if not I'd prefer just to roll my own SQL going forward.
We released a tool for generating Go types/funcs for databases/queries that might help you with this:
https://github.com/knq/xo
(FWIW I completely agree the added mental overhead for ORMs are not necessary)
https://github.com/knq/xo
(FWIW I completely agree the added mental overhead for ORMs are not necessary)
Your tool is exactly what i think makes the most sense so far in having go work with DB. It's type safe (assuming the tool is correct), and doesn't add complexity in the code or complex and magic libraries.
It also makes the database schema the source of truth for a system, which makes the most sense to anyone that has really worked on complex systems (most of them having the DB be accessed by multiple subsystems, written in different languages).
It also makes the database schema the source of truth for a system, which makes the most sense to anyone that has really worked on complex systems (most of them having the DB be accessed by multiple subsystems, written in different languages).
Thanks!
If you're actively using xo, I'd love to know more about how you're using it. I'm in the middle of writing up some real world examples of how to use it, and would love to be able to point to work done by others.
If you're actively using xo, I'd love to know more about how you're using it. I'm in the middle of writing up some real world examples of how to use it, and would love to be able to point to work done by others.
i stopped using go because it didn't have a good enough db access library (i like using ORMs for prototyping, and then see what really needs to be recoded with direct sql later).
Not sure how you did this, but this kind of data layer generation tools were pretty common in the early C# days, you should have a look and see how they worked. The main idea is to keep a "_generated" folder independant from the rest of your code, and let people enrich your models with methods in separate files (so that you can regenerate without breaking anything). But go makes all that pretty easy, so i assume it's already the case.
Another feature is to be able to configure how many foreign relationships are queries at the same time when you request one top object, to be able to optimize db access on the most common paths.
Not sure how you did this, but this kind of data layer generation tools were pretty common in the early C# days, you should have a look and see how they worked. The main idea is to keep a "_generated" folder independant from the rest of your code, and let people enrich your models with methods in separate files (so that you can regenerate without breaking anything). But go makes all that pretty easy, so i assume it's already the case.
Another feature is to be able to configure how many foreign relationships are queries at the same time when you request one top object, to be able to optimize db access on the most common paths.
The composability of queries that an ORM like ActiveRecord seems like too much of a good thing to give up. Getting the same functionality by cutting and pasting bits of SQL together is much more fragile and far less flexible. Soon enough you find yourself abstracting the various parts of the query (select columns, where clauses etc) and you're on the way to building your own ORM already.
I've abandoned ORMs everywhere. SQL is easy and abstraction layers tend to make easy things slightly easier but harder things much harder.
In the early days, qb has only the builder api. However, I also wanted to have a table builder from structs. Currently, the builder has completely separate implementation from session. The session is the structural composition of builder, engine, metadata, etc. You can see more examples in https://qb.readme.io/docs/the-builder
Feedbacks & Contributions are welcome about the builder.
Feedbacks & Contributions are welcome about the builder.
Did you look at the above work (qb) or have any comments specific to it?
Not liking gorm or orm in general is a common enough attitude, but it feels like an unfair dismissal to not be specific about the posted library and instead criticize it based on your experience with another.
I also might have a misread here, but I doubt people are adopting query builders because they find sql hard, but rather due to combinatorial explosion in manual sql building in some application types.
Not liking gorm or orm in general is a common enough attitude, but it feels like an unfair dismissal to not be specific about the posted library and instead criticize it based on your experience with another.
I also might have a misread here, but I doubt people are adopting query builders because they find sql hard, but rather due to combinatorial explosion in manual sql building in some application types.
I agree; it's a little tedious to have to write the boilerplate required, but in the end it works fine. I know exactly what queries my code is running rather than potentially being surprised by what a package/framework may be doing.
The biggest irritation I have is dealing with null values, but the built-in null types in the SQL package are adequate, and there are packages that make them play nice with JSON as well. JSON and array types in Postgres can also be a little cumbersome, but the database packages I've tried don't seem to properly handle them anyway.
The biggest irritation I have is dealing with null values, but the built-in null types in the SQL package are adequate, and there are packages that make them play nice with JSON as well. JSON and array types in Postgres can also be a little cumbersome, but the database packages I've tried don't seem to properly handle them anyway.
An alternative to Go's SQL null types is simply using pointers, like `*string` instead of `sql.NullString`. Pointers function in the same way, except they marshal into JSON very nicely.
I generally have the same feeling, though https://github.com/go-gorp/gorp has worked very well for most of the database-heavy Go projects I've worked on over the last few years. It is just enough abstraction to be convenient and consistent and rarely gets in the way.
When I read the title I thought this would be a toolkit for writing databases.
A lot of the work involved in writing a database is systems stuff such as being sure that a commit log has actually been committed. Having a toolkit that does all that stuff for you would make it a lot easier to write your own database.
A lot of the work involved in writing a database is systems stuff such as being sure that a commit log has actually been committed. Having a toolkit that does all that stuff for you would make it a lot easier to write your own database.
Looks nice! Go needs libraries like these.
If you're interested to see more alternatives to Gorp and Gorm, here is another one; http://github.com/azer/crud
How does it compare to https://github.com/jinzhu/gorm/ ?
The struct tags are grouped like type, constraints, etc. The groups of struct tags is more Gorm currently can't auto migrate keys with foreign keys. There is an issue I've commented but the issue is open for a long time.
There are also some missing features qb doesn't have and gorm has. For instance, you can define relationships in gorm while in qb, you can only define foreign keys using `qb:"constraints:ref(col, table)"`.
Moreover, I am not entirely sure but I don't think enforcing types is possible in gorm. In qb, consider this struct;
type User struct { id string `qb:"type:uuid; constraints:primary_key"` }
qb understands this as a uuid type although it has string definitions. These are the main reasons why I created qb.
The relationship feature is not clear in my mind. I'd like to have feedback on relationships.
There are also some missing features qb doesn't have and gorm has. For instance, you can define relationships in gorm while in qb, you can only define foreign keys using `qb:"constraints:ref(col, table)"`.
Moreover, I am not entirely sure but I don't think enforcing types is possible in gorm. In qb, consider this struct;
type User struct { id string `qb:"type:uuid; constraints:primary_key"` }
qb understands this as a uuid type although it has string definitions. These are the main reasons why I created qb.
The relationship feature is not clear in my mind. I'd like to have feedback on relationships.
Have you considered another mechanism apart from field tags for relating fields to the db (for example a function for marshalling?). It's starting to look like you're creating a dsl stuffed into struct field tags as strings, which gets ugly if it is complex. This is one part of go I'm really not keen on for this reason.
In https://github.com/aodin/sol#sol I separated the table schema, which is a function, from the destination / receiver structs. This approach allows you to build tables programmatically and do condensed selections, such as all IDs into a []int. However, this process still has to match database columns to struct field names for most selections.
There is a table api in qb much like yours. See https://github.com/aacanakin/qb/blob/master/table_test.go. The session api converts tags into table object. I really like the column definitions. Maybe I can convert the definition implementation like you.
I'm also not happy to add tons of tags in struct fields. Can you provide an example on how to do that?
What I understand is to have a function inside the struct namely like "map" and define the constraints there. I think we can do this in the next milestone of qb.
Can you open an issue about this? The github repo is https://github.com/aacanakin/qb
I just use a function taking the row from the db in the model and creating the model from that. It avoids tags and reflection at the cost of some verbosity.
[deleted]
Also, some advice: if you're looking to abstract away from SQL because you find it hard - your time is much better spent learning a time-tested standard than yet another framework that may vanish tomorrow.