Bradley Holt gave a presentation on Apache CouchDB 2.0, which introduced new features such as clustering, shards and replicas, Mango querying, and per-user databases. The presentation demonstrated how to connect to and interact with CouchDB through the HTTP API using the http-console tool, including creating databases and documents, retrieving documents, adding attachments, conditional requests, and deleting documents.
12. Shards and Replicas
q=8
– Number of shards
– One or more shards per node
– Cannot have more nodes than shards
r=2
– Read quorum
w=2
– Write quorum
n=3
– Number of replicas of every document
@BradleyHolt
CouchDB
node1
CouchDB
node2
CouchDB
node3
16. What else is new?
Improved database compaction
Faster index updates
New _bulk_get endpoint for
optimized replication
View-based filters in _changes feed
Filter _changes feed with _doc_ids
_all_docs and _changes will
support attachments=true
@BradleyHolt
37. Connecting to CouchDB 2.0
$ http-console root:passw0rd@127.0.0.1:5984 --json
@BradleyHolt
38. Connecting to CouchDB 2.0
$ http-console root:passw0rd@127.0.0.1:5984 --json
> http-console 0.6.3
> Welcome, enter .help if you're lost.
> Connecting to 127.0.0.1 on port 5984.
@BradleyHolt
39. Connecting to IBM Cloudant
$ http-console https://bradley-holt:passw0rd@bradley-holt.cloudant.com --json
@BradleyHolt
40. Connecting to IBM Cloudant
$ http-console https://bradley-holt:passw0rd@bradley-holt.cloudant.com --json
> http-console 0.6.3
> Welcome, enter .help if you're lost.
> Connecting to bradley-holt.cloudant.com on port 443.
@BradleyHolt
47. PUT a Database
/> PUT /kittens
...
HTTP/1.1 201 Created
Content-Type: application/json
Location: http://127.0.0.1/kittens
{ ok: true }
@BradleyHolt
51. PUT a Database Again
/> PUT /kittens
...
HTTP/1.1 412 Precondition Failed
Content-Type: application/json
{ error: 'file_exists', reason: 'The database could not be created, the file already exists.' }
@BradleyHolt
72. PUT an Attachment
/kittens> PUT /mittens/photo?rev=1-e665a40d9ea9711c983e907f0b0b6e8a
... R0lGODlhAQABAIAAAP///wAAACwAAAAAAQABAAACAkQBADs=
@BradleyHolt
73. PUT an Attachment
/kittens> PUT /mittens/photo?rev=1-e665a40d9ea9711c983e907f0b0b6e8a
... R0lGODlhAQABAIAAAP///wAAACwAAAAAAQABAAACAkQBADs=
HTTP/1.1 201 Created
Content-Type: application/json
Location: http://127.0.0.1/kittens/mittens/photo
{
ok: true,
id: 'mittens',
rev: '2-d858e51453a5785bafe517b7eddc5a98'
}
@BradleyHolt
74. PUT an Attachment
/kittens> PUT /mittens/photo?rev=1-e665a40d9ea9711c983e907f0b0b6e8a
... R0lGODlhAQABAIAAAP///wAAACwAAAAAAQABAAACAkQBADs=
HTTP/1.1 201 Created
Content-Type: application/json
Location: http://127.0.0.1/kittens/mittens/photo
{
ok: true,
id: 'mittens',
rev: '2-d858e51453a5785bafe517b7eddc5a98'
}
/kittens>
@BradleyHolt
75. PUT an Attachment
/kittens> PUT /mittens/photo?rev=1-e665a40d9ea9711c983e907f0b0b6e8a
... R0lGODlhAQABAIAAAP///wAAACwAAAAAAQABAAACAkQBADs=
HTTP/1.1 201 Created
Content-Type: application/json
Location: http://127.0.0.1/kittens/mittens/photo
{
ok: true,
id: 'mittens',
rev: '2-d858e51453a5785bafe517b7eddc5a98'
}
/kittens> Content-Type: application/json
@BradleyHolt
78. GET an Attachment
/kittens> GET /mittens/photo
HTTP/1.1 200 OK
Content-Type: image/gif
Etag: "UsqjdPnY6ApD2ENFOglFHg=="
R0lGODlhAQABAIAAAP///wAAACwAAAAAAQABAAACAkQBADs=
@BradleyHolt
148. Image Credits
paper by malik, on Flickr <https://flic.kr/p/aZjTXv>
person by Tim Morgan, on Flickr <https://flic.kr/p/7DSF5>
database by Tim Morgan, on Flickr <https://flic.kr/p/7DUk5>
gear by Tim Morgan, on Flickr <https://flic.kr/p/7DSF1>
Mango with section on a white background by bangdoll, on Flickr <https://flic.kr/p/9CBP2h>
@BradleyHolt
A comprehensive portfolio of open source data services
A rich, curated open data catalog that adds meaning to your data
Services that connect integrate hybrid data from internal and external data sources and connect to analytics offerings
Fully managed and open for business 24/7/365
Brings horizontal scalability to CouchDB
CouchDB 2.0 can be configured for standalone mode
Can add and remove nodes to a cluster at will
Cannot reshard, so preshard instead
Presharding, no resharding
If you need more shards, create a new cluster and migrate over
enable
If set to true, couch_peruser ensures that a private per-user database exists for each document in _users.
These databases are writable only by the corresponding user.
Databases are in the following form: userdb-{hex encoded username}.
delete_dbs
If set to true and a user is deleted, the respective database gets deleted as well.
The Cloudant and CouchDB 2.0 codebases are merging
Clustering and Mango have been available on Cloudant for quite awhile
HTTP Basic authentication is a simple way to authenticate with an HTTP server
Other approaches, such as cookies and OAuth, are often used as well
Resources can be represented in different content types
…
I hope that I’ve convinced you that you shouldn’t build your apps to rely on a network connection. But what’s the alternative? You have two basic options: graceful degradation or progressive enhancement. Graceful degradation involves building your app as usual, assuming that it will have a good network connection, and then gracefully handling the resultant errors when the network connection inevitably does not work. The progressive enhancement approach involves assuming that, as a baseline, your app will not have a working network connection. Then, your app can be progressively enhanced to take advantage of network connectivity when available.
The progressive enhancement approach is what we call Offline First. Don’t assume that lack of connectivity is an error condition. Build your app to work without a connection. Get updated content, sync your data, or enable features that aren’t practical to make work offline when the app is connected. Let users still interact with your app when there’s no reliable connection available.
The term Offline First was first coined by Alex Feyerke in a 2013 A List Apart article. The open source Hoodie project (which Alex and others work on) was probably the first framework to embrace the Offline First moniker and approach. Around the same time that Hoodie was created PouchDB was also created. PouchDB is a JavaScript database that syncs. More recently, Service Workers were introduced into Google Chrome, enabling new Offline First use cases in web browsers. Firefox and Opera have also implemented Service Workers, and Microsoft Edge plans to support Service Workers as well.
…
If you’ve heard of Progressive Web Apps then some of what I’ve talked about probably sounds familiar. The basic idea of Progressive Web Apps is to combine the discoverability of web apps with the power of native mobile apps. As an end user, you browse to a Progressive Web App just like you would browse to any other website. As you use the app more-and-more, it gains additional native-app-like capabilities. For example, installation to your home screen or the ability to send you alerts and notifications.
Google is currently making a big push for Progressive Web Apps and many companies have invested in building Progressive Web Apps. For example, The Weather Company, which IBM recently acquired, has built a Progressive Web App which allows them to send alerts and notifications to users.
Progressive Web Apps are an opportunity for web developers to rethink how we approach building web apps. The market has clearly demonstrated that it likes the interaction model of native apps. There’s a lot that we can learn from how native apps are built.
Don’t get me wrong, I’m a huge believer in the power of the open web. I don’t think we should just turn web apps into native apps and call it done. I do think that we can incorporate some of the best parts of native apps into how we build web apps while not losing the soul of the open web.
I encourage you to also check out PouchDB. It’s not as lightweight as localForage, but it includes a bunch of awesome features that are probably worth the extra overhead. The most exciting feature of PouchDB is its ability to sync with anything that implements the CouchDB Replication Protocol. This, of course, includes Apache CouchDB. It also includes IBM Cloudant, a fully-managed cloud database that is based on CouchDB. PouchDB allows you to read and write data directly on the device, and then sync this data to the cloud when you have an internet connection. This makes create, read, update, and delete operations extremely fast as all of your data access happens locally.
Not only can PouchDB run in a web browser, but it can also run in any JavaScript environment including in Node.js, in Apache Cordova or Ionic for hybrid mobile apps, in Electron for desktop apps, and even on a Tessel for Internet of Things (IoT) apps. Nolan Lawson, one of the core contributors to PouchDB, has written up some excellent demos of running PouchDB in these environments and others.
Our team at IBM Cloudant also created open source Cloudant Sync libraries for iOS and Android, in case you prefer to build native mobile apps and would like to take an Offline First approach. While they have different APIs than PouchDB, these libraries, like PouchDB, store data locally on the device and can also sync with anything that implements the CouchDB Replication Protocol.
…
Our Developer Advocacy team has built several Offline First sample apps to help you learn how to build Offline First apps. Cloudant FoodTracker is an app that we created to demonstrate how to build an Offline First iOS app in Swift using Cloudant Sync. Apple provides a great tutorial on starting to develop iOS apps in Swift. The tutorial walks readers through creating a simple meal tracking app. The app allows users to view and edit a list of meals. Each meal includes a meal name, rating, and a photo.
Apple’s FoodTracker is offline-only, meaning that all of the meal data is stored only on the device and the app provides no means of syncing that data to the cloud. Cloudant FoodTracker replaces the data layer with Cloudant Sync and then syncs with Cloudant or CouchDB.
Field Work is a web app that allows offline editing and mapping of geospatial data. The idea of Field Work is to demonstrate how a company in a field-based industry, such as the utility industry, could benefit from an app which their field personnel could use even when disconnected.
Field Work stores GeoJSON locally on the device in PouchDB. It uses Leaflet and Mapbox.js to render the geospatial data on a map. Read-only base map layers are synced from a Cloudant database to a PouchDB database on the device. An editable map layer is then synced between a separate Cloudant database and a separate PouchDB database on the device. There are other aspects to the app, including the ability create work orders.
Location Tracker is another iOS sample app, written in Swift. The app tracks a user’s location as they move and stores this data in a local Cloudant Sync database, which is synced with Cloudant when an internet connection is available. As a user moves, and new locations are recorded, the app queries the server for points of interests near the user’s location. Blue pins mark each location recorded by the app. A blue line is drawn over the path the user has travelled. Each time the Location Tracker app records a new location a radius-based geo query is performed in Cloudant to find nearby points of interest.