willholley

10 ปีที่แล้วdiscuss

The chattiness is mostly addressed with _bulk_get in CouchDB 2.0 - Pouch will automatically use it if the server supports it. Another option is to stick a HTTP/2 proxy in front of your CouchDB instance - the chatter to the db is ultimately still there but it significantly reduces the latency cost to the PouchDB client. There are plans to add first class HTTP/2 support to Couch but for remote client architectures just adding a proxy should be a significant improvement. Projects like https://github.com/cloudant-labs/envoy take this a step further and provide an extensible proxy (e.g. you can do sub-database access control, etc).

willholley

10 ปีที่แล้วdiscuss

There are a lot of use cases, particularly around multi-region/DC master-master replication, where CouchDB/Cloudant is a great fit. IBM also offers MongoDB, RethinkDB, Postgres as managed services and the MVCC/conflict model in Couch is still a standout feature. The other use case I see a lot of is offline-capable apps backed by Couch replication-compatible datastores (PouchDB, Cloudant Sync, Couchbase Lite).

Cloudant recently changed its pricing structure [1] (via IBM Bluemix) which should significantly reduce the cost of continuous replication - that may be worth a look if the current model is problematic.

[1] https://developer.ibm.com/bluemix/2016/07/26/cloudant-public...

willholley

10 ปีที่แล้วdiscuss

I think there's a lot for you to like in 2.0.

> The replication protocol, which supports multi-master, has changed little

Basically true, though with many interoperating, independent datastores it's a tricky thing to evolve. 2.0 adds an additional endpoint, _bulk_get which can significantly reduce the number of requests when CouchDB is paired with an on-device database such as PouchDB, Cloudant Sync or Couchbase Lite (the endpoint was inspired by the same feature in Couchbase). The CouchDB replicator itself has had many performance and stability improvements [1] and continues to be a significant focus for active development.

Also, CouchDB 2.0 introduces internal cluster replication using distributed Erlang. If you currently use CouchDB replication to bi-directionally replicate between machines on the same network for HA, replacing those with a CouchDB 2.0 cluster should be a big win.

> In other words: everybody seem to be looking at CouchDB as just a very poor and limited MongoDB.

Query doesn't pretend to be MongoDB-compatible - it provides a syntax that should be familiar to MongoDB users and more query flexibility than views allow. I think Query still has a fair way to go - this is the first release - but it's a move in the right direction.

As to whether CouchDB is viewed as "a very poor and limited MongoDB", they are very different databases. CouchDB is a good choice if you want a rock-solid JSON datastore which comfortably scales up to multiple TBs / many machines, with multi-master replication over unreliable networks. Query support, as you say, is not as rich as some other databases, so if that's more important to you, there are probably better options.

> Filtered replication was implemented, but it is slow to the point that no one recommends that you use them.

The new _selector filter [[2] in CouchDB 2.0 offers a significant performance improvement for filtered _changes. It should be a small change for replicators such as PouchDB can take advantage of this.

> About Couchapps, the special database features that powered them in the first place were left aside

I don't speak for the project, but it seems there has been much debate about this in the CouchDB community and the conclusion was that there are better solutions to most Couchapp-shaped problems than running application logic in the database. The features that combine to enable Couchapps haven't gone away and will benefit from the general improvements in 2.0, but they haven't been explicitly developed.

[1] https://blog.couchdb.org/2016/08/15/feature-replication/ [2] http://docs.couchdb.org/en/2.0.0/api/database/changes.html#s...

willholley

11 ปีที่แล้วdiscuss

possibly this has changed recently but I don't think medical records software is subject to the same standards as medical device software (in Europe, at least).

It's important to remember that EMRs are there to support healthcare professionals who are used to working with incomplete / incorrect information all the time. For example, a patient isn't going to have a drug dispensed to them without a pharmacist checking it (and, frequently, asking the prescribing doctor to clarify/amend questionable doses or instructions).

willholley

11 ปีที่แล้วdiscuss

For MongoDB lovers, PouchDB supports MongoDB-style querying via https://github.com/nolanlawson/pouchdb-find. MongoDB doesn't have the fundamental sync primitives that CouchDB has so I wouldn't expect a similar solution which supports MongoDB on the server.

willholley

12 ปีที่แล้วdiscuss

That's fine for scaling reads - writes, not so much. The leader election in MongoDB has been problematic in the past (https://aphyr.com/posts/284-call-me-maybe-mongodb) but I think they have made recent moves to address this.

willholley

12 ปีที่แล้วdiscuss

ES isn't really meant to be used as an operational datastore (although I appreciate some do use it that way). CouchDB provides strong durability and master-master replication. It's also very easy to sync changes from CouchDB (https://github.com/elasticsearch/elasticsearch-river-couchdb) to ES so they make a good complimentary pair. Use CouchDB for the raw data (where you want strong durability and write scalability) and use ES to search over it.

willholley

12 ปีที่แล้วdiscuss

Although it's been a quiet few years, the project is on the up again. 2014 was a big year for CouchDB - CouchDB 2.0 developer preview was released (more on this in a sec) [1], IBM acquired Cloudant (and has made it available as on-premise offering), Cloudant Query (MongoDB-style querying for CouchDB)[2] was contributed to CouchDB. PouchDB, Couchbase Mobile and Cloudant Sync (all replication-compatible with CouchDB) all saw major new releases and increased uptake. Much more detail on all of this is on the CouchDB blog [3].

CouchDB 2.0 adds dynamo-style clustering support (similar to Riak and Cassandra) in addition to the replication(sync) protocol used by the CouchDB ecosystem [4]. It also includes all the fixes / performance improvements based on Cloudant's operational experience over the last 5 years.

That said, document databases are certainly a niche. CouchDB is a good choice if you need strong durability (writes are always fsync'd - to multiple copies when clustering) and consistent performance as your database scales (querying options may seem restrictive but are designed to scale very well). As others have pointed out, the RESTful interface to CouchDB makes it a good fit for web and mobile applications which can query the database directly without an app tier. PouchDB [5] and various mobile datastores which implement the sync protocol [6][7] allow you to also take your database to a browser or mobile client and work with it when disconnected which is pretty compelling.

[1] https://speakerdeck.com/wohali/putting-the-c-back-in-couchdb... [2] https://cloudant.com/blog/introducing-cloudant-query [3] http://blog.couchdb.org/2014/12/19/couchdb-weekly-news-decem... [4] http://www.replication.io/ [5] http://pouchdb.com/ [6] https://cloudant.com/cloudant-sync-resources [7] http://www.couchbase.com/nosql-databases/couchbase-mobile

willholley

12 ปีที่แล้วdiscuss

database-per-user is a common pattern with CouchDB. If you need to aggregate data across your users for reporting etc then you can use a server-side replication to create a "master" database which is only accessible to your internal users.

You could also use filtered replication but it is: a) not scalable b) not secure (the whole db would still be accessible to all users)

Couchbase has a different spin on this via "channels" though I'm not sure how it interoperates with PouchDB.

willholley

12 ปีที่แล้วdiscuss

We (Cloudant) ran a Jepsen-style test against our clustered CouchDB implementation (similar to BigCouch). The write-up is at https://cloudant.com/blog/run-dmc-explains-network-partition....

The clustering layer is getting merged into CouchDB and will provide semantics closer to other Dynamo-style systems (e.g. Cassandra, Riak).

willholley

12 ปีที่แล้วdiscuss

shameless plug: Cloudant Geo (https://cloudant.com/product/cloudant-features/geospatial/) uses a sharded R*-Tree for big geospatial data.

willholley

13 ปีที่แล้วdiscuss

Have you looked at Cloudant (https://cloudant.com/)?

willholley

13 ปีที่แล้วdiscuss

True - multi-master in CouchDB means that you can have two CouchDB instances (or, indeed, anything which speaks the CouchDB replication protocol - see http://www.replication.io/) which can be synced and, in the event of a network partition, both instances can be written to. One of those masters could be CouchDB and one could be PouchDB in a browser or TouchDB on a mobile phone. Neither instance requires any coordination from the other.

Clustered CouchDB (BigCouch - on it's way to being integrated into CouchDB vNext) relies on the same MVCC semantics - it's just using Erlang rather than HTTP to transfer documents between clusters and attempts to keep the nodes in sync continuously.

Conflict resolution is tricky but CouchDB plays it safe and keeps all conflicting revisions of each document around until you resolve them to ensure no data is lost. It's pretty easy to find and fetch any conflicted documents so they can be resolved in an application-specific way.

willholley

13 ปีที่แล้วdiscuss

Bluewire Technologies - Bristol, UK http://www.bluewire-technologies.com/

We are a small (currently 6), established team building innovative software for health professionals in the UK. There are many interesting challenges around data modelling, UI / UX, scaling, mobile, security - it's an exciting space to be in! We're passionate about providing healthcare workers with world-class clinical software, and we're in it for the long-term. The working environment is relaxed, but we get things done. We'll give you great kit and support you working from home as and when you want to. Salary is competitive and depends on the candidate.

We're always interested to hear from any talented engineers, but specifically we have a need for:

A front-end / mobile engineer. You would suit this position if:

  - You've worked on at least one web project of significant complexity.
  - You know what jQuery, backbone and SASS are and when they are useful.
  - You know how to architect for performance and can reason about trade-offs.
  - You are already building mobile web applications or keen to learn.
  - You like to take responsibility for UI design and improving the user experience.
  - You've led a small development team before or have an interest in taking a step in that direction.

A lead devops / sysadmin:

You'd be responsible for deploying, monitoring and tuning our web-application on customer servers - currently 40 servers over 14 sites with an expectation that this will increase significantly in the next 18 months. At the moment this task is performed by a member of the development team and we are now reaching a scale where we need somebody in a dedicated role. You'd be responsible for scheduling and deploying builds to staging and production environments, "owning" the the release management processes and working with our developers to troubleshoot performance / reliability problems as they arise. You'd suit this position if:

  - You understand how to administer SQL Server and use the performance profiling tools.
  - You can troubleshoot IIS and understand how it interacts with ASP.NET. 
  - You know how to use Windows performance counters to diagnose problems.
  - You're a methodical problem solver and enjoy working with customers.
  - You appreciate the relationships between people and task automation.

If this sounds like you, shoot me an email: [email protected]

willholley

14 ปีที่แล้วdiscuss

Just for clarity, the minimum 28 days in the UK is inclusive of the 8 bank holidays.

willholley

15 ปีที่แล้วdiscuss

There was a recent statement on the plans to open up more data sets - see http://www.cabinetoffice.gov.uk/sites/default/files/resource...

willholley

15 ปีที่แล้วdiscuss

One problem is that as a vendor, we have to work to the lowest common denominator. It's still generally a requirement that you support IE6, despite the department of health encouraging hospitals to upgrade, because they have legacy apps where the vendor has disappeared or the upgrade is too expensive / painful for the hospital to bear.

We actually ship customers a custom build of Chromium which can only access our application (hospital policy generally dictates they must use IE to access the internet for "security" reasons) - we would love them to just use modern browsers so we can ditch our custom client but it doesn't look like it's going to happen any time soon :-(

willholley

16 ปีที่แล้วdiscuss

As a vendor in the UK, we have been integrating the CUI for the last few years. The quality of the guidance is mixed - there is good evidence that having a "Patient Banner" in a consistent format prevents "right thing, wrong patient" problems. However, a lot of the evidence supporting the other guidance is sketchy. They also lean heavily on MS implementations of the guidance which I would think most vendors will avoid because it does not likely fit into their technology stack (Silverlight / WPF).

However, the real problem is that the market does not value good usability. We are lucky if a potential customer has even heard of the Common User Interface, let alone specifying it as a requirement on a tender.

The "ghetto" effect described by another poster is largely down to the "market for lemons" around healthcare IT, in the UK at least (many different vendors promising a silver bullet to hospitals with no experience in procuring such things - they just pick the cheapest / "least risky").

willholley

16 ปีที่แล้วdiscuss

Very similar to the analysis by Steven Murdoch and Ross Anderson published in January: see http://www.cl.cam.ac.uk/~rja14/Papers/fc10vbvsecurecode.pdf

willholley

16 ปีที่แล้วdiscuss

It is possible to work in an agile manner with the NHS but it is not easy. I have been working for a small company selling software into NHS hiospitals for 7 years now, and we generally take a collaborative approach with customers when developing new functionality (which we can then upsell to our other customers). We have a progressive development environment with a small but talented development team.

However, I believe that the market for healthcare software in the UK is severely flawed, and there is little commercial incentive for companies to produce high quality products. Specifically,

1. As identified elsewhere on this thread, the procurement process generally does not reward quality or functionality above accountability / company financials.

2. The cost of sales is very high, and hospital trusts have no money. Therefore, suppliers will generally sell at a loss and require significant revenue through support contracts to make the contract viable. The business is all about lock-in, and there is very little customer demand for real interoperability between systems. Startups are unlikely to become profitable for a long time. It also means that customers resent suppliers for charging through the nose for an apparently trivial change to the software.

3. For any non-trivial IT solution, there are significant interoperability and business process differences between hospitals. You end up implementing a custom solution for each trust and have to absorb the operational costs associated with this. As a supplier, we spend a lot of time trying to get trusts to talk to each other to agree on standard processes, but really this needs to be done at a national level.

4. Many hospitals will roll their own solutions because it is "cheaper". A huge number of hospital systems have been written by doctors who dabble in programming and grow into a big ball of mud. Indeed, our product started out like this (although the doctor in question was smart enough to hire some programmers early on). For me, this shows the lack of experience in the IT departments which allow this to happen, as it is generally not the right way to go.

There are many other challenging aspects to this market, but it does feel like there are opportunities for good software companies to make a difference.

comments