Bookmark services are a dime a dozen these days. When your career and hobbies require you to have hundreds of links saved, things tend to get messy. I eventually settled on Diigo because of its support for both lists and tags – you can add multiple tags to every bookmark, and you can add every bookmark to a list. But visiting these bookmarks is tedious – I first have to open my Diigo library in a new tab, and then click on the bookmark in the list before me. If my library is complex and deeply nested, all the more trouble – I need to further filter my search by clicking the filters on the left hand side, and I'm already spending much more time trying to get to my bookmarked site than I should.
Planning
In this series, we'll create a Google Chrome extension which hooks up to the Diigo API, retrieves the bookmarks saved there, and syncs them into a folder on the Chrome bookmarks bar. This folder will have several levels:
- Root level. A subfolder called "Tags" will be here, as well as all the bookmarks the user tags as bbs-root.
- Tags level. The "tags" subfolder will contain one folder for every tag the user has in their Diigo library. Entering said folder lists all the posts with the given tag.
Unfortunately, since Diigo's API is rather underdeveloped, there is no way to delete tags should they be left empty, nor is there a way to delete a bookmark from Diigo if it gets deleted in Chrome – yet. If this API shows up, I'll leave it to someone else to write a followup article. Likewise, the Diigo API does not support lists as of this moment. Once they add this functionality, it should be straightforward enough to upgrade this extension with a "Lists" subfolder.
It's important to note that Google is very monopolistic with its bookmarks service. Chrome has a maximum write limit built in, meaning you cannot do more than 100 writes (create, update and delete) via the chrome.bookmarks API per hour. What this means is that if someone has more than 100 tags/lists/bookmarks in Diigo, their browser will take several hours before fetching them all and eventually settling for fewer writes (only updates, creates and deletes from that point onward should be far less common). We'll also be using JavaScript 1.7 constructs like the let keyword, so you should go into chrome://flags and enable "Experimental JavaScript". Could it be done without let? Absolutely. But I firmly believe that staying away from new technology just because it's not everywhere yet is harmful to both developers and the web in general. JS 1.7 came out 7 years ago, which is something like three centuries in internet years. In addition to let, we'll be using "strict mode", because let cannot be used without it.
Note that this means people without experimental JS enabled won't be able to install and use this extension, at least until JS 1.7 support is enabled by default in Chrome.
Bootstrapping
First, let's create a folder in which we'll hold our extension's source code. Create a folder structure such as this one, and leave the JS and JSON files blank.
/
icons/
background.js
manifest.jsonWhat we need next is the manifest.json file filled out.
{
"name": "Diigo Bookmark Bar Sync",
"description": "Sync Diigo Bookmarks to Chrome",
"version": "1.0.0.0",
"background": {
"scripts": ["background.js"]
},
"permissions": [
"bookmarks", "https://secure.diigo.com/api/v2/"
],
"browser_action": {
"default_icon": {
"19": "icons/19.png",
"38": "icons/38.png"
},
"default_title": "Diigo BBS"
},
"icons": {
"16": "icons/16.png",
"48": "icons/48.png",
"128": "icons/128.png"
},
"manifest_version": 2
}If you've followed along with my previous Chrome Extension tutorial on Sitepoint, you should be familiar with all the keys and their values.
There are three novelties you might not be familiar with: the fact that we're using a JS background page instead of HTML (irrelevant either way – JS is unnoticeably faster), we're requesting the "bookmarks" permission to ask Chrome to let us edit them, and we're requesting permission to access https://secure.diigo.com/api/v2/ which helps us with cross origin ajax or, in other words, lets us do Ajax calls on Diigo without raising security flags.
We're also using a browser_action, which means we'll have a persistent icon NEXT to our omnibar at all times – not inside it while we're on a specific page, as is the case with page actions.
Make some icons for your extension in sizes mentioned in the manifest.json file and add them to the icons folder, or just download mine and put them there.
At this point, we can test our extension by loading it into the extensions tab (chrome://extensions). Make sure "Developer Mode" is checked, and click "Load Unpacked Extension", then point Chrome to the folder where you've put the files. If everything goes well, the extension's icon should appear in the top bar to the right of the omnibar and if you mouse over it, you should see "Diigo BBS" pop up.
Diigo API
To gain access to Diigo's API, you need to sign up for an API key. This will provide you with a string of random characters which you need to send along with every Diigo API request in order to identify yourself (actually, in order to identify your app – every app will have a different API key).
Diigo's API is severely underdeveloped, but RESTful which means we call the same URL for acting on the same object (i.e. Bookmarks) every time, but change the request type (GET fetches, POST updates and inserts, DELETE deletes the bookmark – not yet implemented). We'll explain this into a bit more depth soon.
Essentially, communicating with the API is as simple as sending a request to the URL, filled with the required parameters. If we assume there's a user called "Joel", to fetch 10 of Joel's bookmarks, we use https://secure.diigo.com/api/v2/bookmarks?key=your_api_key&user=joel&count=100&filter=all
The response to this request will either be an error code if something went wrong, or a JSON object. This JSON object will either contain nothing if Joel has no bookmarks, or will contain data blocks corresponding to information on those bookmarks, much like the example in the API docs demonstrates:
[
{
"title":"Diigo API Help",
"url":"http://www.diigo.com/help/api.html",
"user":"foo",
"desc":"",
"tags":"test,diigo,help",
"shared":"yes",
"created_at":"2008/04/30 06:28:54 +0800",
"updated_at":"2008/04/30 06:28:54 +0800",
"comments":[],
"annotations":[]
},
{
"title":"Google Search",
"url":"http://www.google.com",
"user":"bar",
"desc":"",
"tags":"test,search",
"shared":"yes",
"created_at":"2008/04/30 06:28:54 +0800",
"updated_at":"2008/04/30 06:28:54 +0800",
"comments":[],
"annotations":[]
}
]It's easy to extract everything we need from this JSON data once we receive it, but we'll get to that in a minute.
The API docs say
The authentication uses HTTP Basic authentication – a standard authentication method that includes base64 encoded username and password in the Authorization request header.
.. but there is neither an explanation nor a demo of this.
It means the following: when you access the actual URL for the API in the browser try clicking this, you get prompted for a username and password.
If you fail to enter the proper credentials, you get a 403 response, which means you have insufficient access.
If you do have the proper credentials, you can access the URL in two ways: either punch them in and submit the form, or include them in the URL, like so: https://myusername:mypassword@secure.diigo.com/api/v2/bookmarks?key=your_api_key&user=joel&count=100&filter=all where myusername and mypassword should be replaced by your information respectively. You can even test this right now in your browser if you registered for an API key and have a valid Diigo account. You should get either an empty array ([]) or a list of your bookmarks (or the public bookmarks of the user you've defined in the user parameter of the URL).
So what does base64 encoding it mean? It means we need to run the username and password through an additional filter, just to account for any weird characters in the password. The string myuser:mypass will thus be converted to bXl1c2VyOm15cGFzcw== (test it here).
So how do we put all this together?
Encoding and sending
First we'll need a way to base64 encode a string. Seeing as JS doesn't have this functionality built in, we can use the code from Webtoolkit. Paste that code into your background.js file. If you want, you can even minify it to make it more compact.
Next, we need to tell the API URL we want to Authorize. This is done with an Authorize header, and when using native XHR objects for Ajax, we can do this with the xml.setRequestHeader('Authorization', auth); method, where auth is a string containing authorization data.
Let's make a common function that generates this auth string.
function make_basic_auth(user, password) {
var tok = user + ':' + password;
var hash = Base64.encode(tok);
return "Basic " + hash;
}As you can see, the returned string will be "Basic " + whatever was calculated from user+pass string as the Base64 value. This string is what the Authorization header needs in order to gain access to the URL we'll be sending it to. It is, essentially, identical to you punching in your username and password manually when you access the URL through the browser.
You might be wondering – couldn't we just add user:pass to the beginning of the URL like we can in the browser, as well, and just ignore the Base64 business? Yes, but then you aren't accounting for misc characters and might run into some serious trouble with invalid requests – for example, the "@" symbol denotes the beginning of the server address and having it in the password would throw a wrench into our efforts.
Finally, let's make an XHR request to the API.
var auth = make_basic_auth('user','pass');
var url = 'https://secure.diigo.com/api/v2/bookmarks?key=your_api_key&user=desireduser&count=100&filter=all';
xml = new XMLHttpRequest();
xml.open('GET', url);
xml.setRequestHeader('Authorization', auth);
xml.send();
xml.onreadystatechange = function() {
if (xml.readyState === 4) {
if (xml.status === 200) {
console.log(xml.responseText);
} else {
console.error("Something went wrong!");
}
}
};Of course, replace "user", "pass", "your_api_key" and "desireduser" with your values.
If we reload our extension now with an open background page (click _generated_background_page.html in the extensions screen to see the background page and console error reports (if any) for our extension), we should see everything is working fine – i.e. there should be no errors in the console of the background page, and there should either be "[]" (an empty array) or something like the following figure:
Conclusion of Part 1
In this part, we've bootstrapped our extension, explained, implemented and demonstrated the Diigo API call. In Part 2, we'll write the bulk of our extension.