The need

  I don't know about you, but I've been living with ad blockers from the moment they arrived. Occasionally I get access to a new machine and experience the Internet without an ad blocker and I can't believe how bad it is. A long time ago I had a job where I was going by bike. After two years of not using public transport, I got in a bus and had to get out immediately. It smelled so bad! How had I ever used that before? It's the same thing.

  However, most of the companies we take for granted as pillars of the web need to bombard us with ads and generally push or skew our perceptions in order to make money, so they find ways to obfuscate their web sites, lock them in "apps" that you have no control of or plain manipulate the design of the things we use to write code so that it makes this more difficult.

Continuous war

  Let me give you an example of this arms race. You open your favorite web site and there it is, a garish, blinking, offending, annoying ad that is completely useless. Luckily, you have an ad blocker so you select the ad, see that it's enclosed in a div with class "annoyingAd" and you make a blocking rule for it. Your web site is clean again. But the site owner realizes people are not clicking on the ad anymore, so he dynamically changes the class name to something different every time you open the page. Now, you could try to decipher the JavaScript code that populates the div and write a script to get the correct class, but it would only work for this web site and you would have to know how to code in JavaScript and it would take a lot of effort you don't want to spend. But then you realize that above the horrid thing there is a title "Annoying ad you can't get rid of", so you write a simple thing to get rid of the div that contains a span with that content. Yay!

  At this point you already have some issues. The normal way people block ads is to create a quasi CSS rule for an element. Yet CSS doesn't easily let's you select elements based on the inner text or to select parents of elements with certain characteristics. In part it's a question of performance, but at the same time there is a matter of people who want to obfuscate your web site taking part in the decision process of what should go in CSS. So here, to get the element with a certain content we had to use something that expands normal CSS, like the jQuery syntax or some extra JavaScript. This is, needless to say, already applicable to a low number of people. But suspend your disbelief for a moment.

  Maybe your ad blocker is providing you with custom rules that you can make based on content, or you write a little script or even the ad blocker people write the code for you. So the site owner catches up and he does something: instead of having a span with the title, he puts many little spans, containing just a few letters, some of them hidden visually and filled with garbage, others visible. The title is now something like "Ann"+"xxx"+"oying"+"xxx"+" ad", where all "xxx" texts appear as part of the domain object model (the page's DOM) but they are somehow not visible to the naked eye. Now the inner text of the container is "Annxxxoyingxxx ad", with random letters instead of xxx. Beat that!

  And so it goes. You need to spend knowledge and effort to escalate this war that you might not even win. Facebook is the king of obfuscation, where even the items shared by people are mixed and remixed so that you cannot select them. So what's the solution?

Solution

  At first I wanted to go in the same direction, fight the same war. Let's create a tool that deobfuscates the DOM! Maybe using AI! Something that would, at the end, give me the simplest DOM possible that would create the visual output of the current page and, when I change one element in this simple DOM, it would apply the changes to the corresponding obfuscated DOM. And that IS a solution, if not THE solution, but it is incredibly hard to implement.

  There is another option, though, something that would progressively enhance the existing DOM with information that one could use in a CSS rule. Imagine a small script that, added to any page, would add attributes to elements like this: visibleText="Annoying ad" containingText="Annxxxoingxxx ad" innerText="" positionInPage="78%,30%-middle-right" positionInViewport="78%,5%-top-right". Now you can use a CSS rule for it, because CSS has syntax for attributes equal to, containing, starting or ending with something. This would somewhat slow the page, but not terribly so. One can use it as a one shot (no matter how long it takes, it only runs once) or continuous (where every time an element changes, it would recreate the attributes in it and its parents).

Feedback

  Now, I have not begun development on this yet, I've just had this idea of a domExplainer library that I could make available for everybody. I have to test how it works on difficult web sites like Facebook and try it as a general option in my browser. But I would really appreciate feedback first. What do you think? What else would you add to (or remove from) it? What else would you use it for?

  A while ago I had discovered the Brave browser for mobiles, which was a Chromium fork that featured ad blocking and privacy guards out of the box, when Google stubbornly refused to enable extension support for mobile Chrome. The thing is, it was only available for mobiles. And even if it were available for the desktop, would it really have anything over Chrome with extensions like uBlock Origin installed?

  The answer is YES! Brave for PC is available and, from the limited interaction I've had so far, it is superior to Chrome. Why? Let me list the reasons:

• it cares about your privacy and not about how Google can track you best, which might not be high on your agenda or on your browser extension creator's agenda. That's a plus, because as you didn't care about it before, you don't have to care about it now, but it's taken care of.
• by removing unnecessary functionality from Chromium, it is actually faster than Chrome! Are you old enough to remember when Chrome appeared as the underdog and Internet Explorer reigned supreme and then everybody was like "IE sucks, Chrome rules because it's so fast and only cares about the user experience"? That's what Brave does now to Chrome!
• it has a "forced" dark mode flag that can turn EVERY web site dark. It's not perfect, but it's out of the box! All you have to do is go to brave://flags/#enable-force-dark and enable the feature. (admittedly, you can achieve the same effect with chrome://flags/#enable-force-dark or edge://flags/#enable-force-dark, but I only found out about it from Brave)
• the best feature yet is the Simplified View. Most of the times when you open a web site in the mobile version, you get a "Show simplified view" button. You click on that and you get:
  
  • just the text of the web site
  • whatever font you want
  • whatever theme you want (dark/bright)
  • no ads
  • no flashing things
  • no sidebars
  • no "accept cookies" and "register/subscribe" popups
    Again, this is probably a Chrome feature, but Brave made it public, visible and natural. Haven't found the way to turn it on the desktop browser, yet.

Correction: I have found that in Chrome you can enable reader mode with a flag (chrome://flags/#enable-reader-mode) only it doesn't work that well. In Brave all that moved to what is called Speedreader, which also must be enabled via a flag (chrome://flags/#brave-speedreader).

  Have you ever browsed one of these "modern" web sites and you got half way through scrolling past the huge image that fills the screen only to get a big popup about registering to some bullshit service, with another popup asking you to enable cookies and then some lazy overlay hiding the content and demanding you pay for the content? Imagine you have one button to click and you get to read the actual information on that page! How can people browse on the web without Brave?!

  I know that some extensions cover most of the points above, but Brave plus uBlock Origin are amazing! I get to a web page that is automatically stripped of most ads, but there are still parts of it that are not strictly ads, like a subscribe form in the middle of the content, for example. You use the Block Element feature of the ad block extension and you get the cleanest browsing experience you can get. (BTW, Brave also has its own Block Element option, so you might not need an extension at all!)

  And there are dark clouds on the horizon. With the V3 manifest version that Google is pushing, many of the APIs available to ad blocker extensions are limited or downright broken. It's not in their interest to block ads, considering they are in the advertising business. Their biggest achievement (and mistake) was to open source Chromium, so no one can take something like Brave away from you.

  Bottom line: switch to Brave. It's like Chrome, only a lot better! And I am not really one of those "fuck the system, stick it to the man" people, so don't think I do this because I have some big agenda. I really really enjoy using this browser and I hope you will, too.

  I guess I don't have to tell you about ad blockers and browser extensions that improve YouTube. They are a dime a dozen and bring many features to the habitual YouTube watcher. However there is one particular new YouTube annoyance that you don't really need an extension to get rid of: the dreaded Video paused dialog.

  To get rid of it is easy: on an interval, check if there is a visible element of a certain type containing a certain text and click its button. While this can be done in simple Javascript, I am lazy, so the script that I am using will first load jQuery, then run a one line function periodically. This code is to be copied and pasted in the Console tab of the browser's development tools, then press Enter.

const scr = document.createElement('script');
scr.setAttribute('src','https://code.jquery.com/jquery-3.4.1.min.js');
document.querySelector('head').appendChild(scr);
setInterval(()=> { 
  $('#button:contains(Yes)','yt-confirm-dialog-renderer:visible:contains(Video paused)').click();
},500);

It's easy to understand:

• create a script element
• set its source to jQuery
• append it to the page
• execute every 500 milliseconds a code that:
  • finds the element with id button containing the text "Yes"
  • inside an element of type yt-confirm-dialog-renderer which is visible and contains the text "Video paused"
  • click the element

There is an even more comfortable solution, though, that I recommend. You will need a Chrome extension called cjs that loads whatever script you tell it in whatever page you want. It gives you the option to inject jQuery, so all you have to do is write 

setInterval(()=> { $('#button:contains(Yes)','yt-confirm-dialog-renderer:visible:contains(Video paused)').click(); },500);

 as the script to be executed on YouTube.

That's it. You're all done.

I am sure I've tested this, but for some reason the icons in my blog disappeared for Internet Explorer. They are using Font Awesome SVG background images, declared something like this:

.fas-comment {
    background-image: url("data:image/svg+xml;utf8,<svg height='511.6' version='1.1' viewBox='0 0 511.6 511.6' width='511.6' x='0' xml:space='preserve' xmlns='http://www.w3.org/2000/svg' y='0'><g fill='#2f5faa'><path d='M477.4 127.4c-22.8-28.1-53.9-50.2-93.1-66.5 -39.2-16.3-82-24.4-128.5-24.4 -34.6 0-67.8 4.8-99.4 14.4 -31.6 9.6-58.8 22.6-81.7 39 -22.8 16.4-41 35.8-54.5 58.4C6.8 170.8 0 194.5 0 219.2c0 28.5 8.6 55.3 25.8 80.2 17.2 24.9 40.8 45.9 70.7 62.8 -2.1 7.6-4.6 14.8-7.4 21.7 -2.9 6.9-5.4 12.5-7.7 16.9 -2.3 4.4-5.4 9.2-9.3 14.6 -3.9 5.3-6.8 9.1-8.8 11.3 -2 2.2-5.3 5.8-9.9 10.8 -4.6 5-7.5 8.3-8.8 9.9 -0.2 0.1-1 1-2.3 2.6 -1.3 1.6-2 2.4-2 2.4l-1.7 2.6c-1 1.4-1.4 2.3-1.3 2.7 0.1 0.4-0.1 1.3-0.6 2.9 -0.5 1.5-0.4 2.7 0.1 3.4v0.3c0.8 3.4 2.4 6.2 5 8.3 2.6 2.1 5.5 3 8.7 2.6 12.4-1.5 23.2-3.6 32.5-6.3 49.9-12.8 93.6-35.8 131.3-69.1 14.3 1.5 28.1 2.3 41.4 2.3 46.4 0 89.3-8.1 128.5-24.4 39.2-16.3 70.2-38.4 93.1-66.5 22.8-28.1 34.3-58.7 34.3-91.8C511.6 186.1 500.2 155.5 477.4 127.4z'/></g></svg>");
}

I had to try several things, but in the end, I found out that there are three steps in order to make this compatible with Internet Explorer (and still work in other browsers):

1. The definition of the utf8 charset must be explicit: data:image/svg+xml;charset=utf8 instead of data:image/svg+xml;utf8
2. The SVG code needs to be URL encoded: so turn all double quotes into single quotes and then replace < and > with %3C and %3E or use some URL encoder
3. The colors need to be in rbg() format: so instead of fill='#2f5faa' use fill='rgb(47,95,170)' (same in style tags in the SVG, if any)

So now the result is:

.fas-comment {
    background-image: url("data:image/svg+xml;charset=utf8,%3Csvg height='511.6' version='1.1' viewBox='0 0 511.6 511.6' width='511.6' x='0' xml:space='preserve' xmlns='http://www.w3.org/2000/svg' y='0'%3E%3Cg fill='rgb(47,95,170)'%3E%3Cpath d='M477.4 127.4c-22.8-28.1-53.9-50.2-93.1-66.5 -39.2-16.3-82-24.4-128.5-24.4 -34.6 0-67.8 4.8-99.4 14.4 -31.6 9.6-58.8 22.6-81.7 39 -22.8 16.4-41 35.8-54.5 58.4C6.8 170.8 0 194.5 0 219.2c0 28.5 8.6 55.3 25.8 80.2 17.2 24.9 40.8 45.9 70.7 62.8 -2.1 7.6-4.6 14.8-7.4 21.7 -2.9 6.9-5.4 12.5-7.7 16.9 -2.3 4.4-5.4 9.2-9.3 14.6 -3.9 5.3-6.8 9.1-8.8 11.3 -2 2.2-5.3 5.8-9.9 10.8 -4.6 5-7.5 8.3-8.8 9.9 -0.2 0.1-1 1-2.3 2.6 -1.3 1.6-2 2.4-2 2.4l-1.7 2.6c-1 1.4-1.4 2.3-1.3 2.7 0.1 0.4-0.1 1.3-0.6 2.9 -0.5 1.5-0.4 2.7 0.1 3.4v0.3c0.8 3.4 2.4 6.2 5 8.3 2.6 2.1 5.5 3 8.7 2.6 12.4-1.5 23.2-3.6 32.5-6.3 49.9-12.8 93.6-35.8 131.3-69.1 14.3 1.5 28.1 2.3 41.4 2.3 46.4 0 89.3-8.1 128.5-24.4 39.2-16.3 70.2-38.4 93.1-66.5 22.8-28.1 34.3-58.7 34.3-91.8C511.6 186.1 500.2 155.5 477.4 127.4z'/%3E%3C/g%3E%3C/svg%3E");
}

After my disappointment with the Firefox for Android lack of proper bookmarks API implementation, I was at least happy that my Bookmark Explorer extension works well with Firefox for desktop. That quickly turned cold when I got a one star review because the extension did not work. And the user was right, it didn't! One of the variables declared in one of the Javascript files was not found. But that only happened in the published version, not the unpacked one on my computer.

Basically the scenario was this:

1. Load unpacked (from my computer) extension
2. Test it
3. Works great
4. Make it a Zip file and publish it
5. Shame! Shame! Shame!

Long story short, I was loading the Javascript file like this: <script src="ApiWrapper.js"></script> when the name of the file was apiWrapper.js (note the lowercase A). My computer file system is Windows, couldn't care less about filename casing, while the virtual Zip filesystem probably isn't like that, at least in Firefox's implementation. Note that this error only affected Firefox and not Chrome or (as far as I know - because it has been 2 months since I've submitted the extension and I got no reply other than "awaiting moderation") Opera.

I've decided the functionality of the Bookmark Explorer extension was pretty close to final and, before I refactor it to a new form, I wanted to make sure it works for all the browsers that support the WebExtensions mechanism, mainly Chrome, Firefox and Opera. Frankly, I have no idea why anyone would use Firefox or Opera, but if you do, I've got great news for you: I have published the extension for all of them:

• Chrome
• Firefox
• Opera

Haven't tested extensively, I am going to do that in the near future, but rejoice, now you can read your news at speed and comfort, then remove them from your bookmarks once you have grown tired. There are some changes to the extension that need to be addressed:

• The most significant is changing the keyboard shortcut for "Previous Bookmark" to Ctrl-Shift-O for Firefox and Opera, because changing extension key shortcuts in Firefox is really difficult and Ctrl-Shift-K is already used by the developer tools
• The default settings have been updated. Now, when you install the extension for the first time you will get:
  • 30 second wait for the "Read Later" links to autoclose, giving the browser time to cache the title and icon
  • Preload next tab is now true by default, leading to loading the content of the next news item while you read the current one
  • When creating bookmarks - from anywhere - their URLs will be stripped of some marketing bullshit
• A lot of bug fixes and speed improvements went into this aparrently minor release

I also plan to make a video of how to use the extension, since letting users read the long description and figure out what the extension does didn't quite work :) I am considering changing the name of the extension for version 3 and I am open to suggestions. I am thinking of Bookmark Surf or something like this. Please let me know of any problems with the extension. I will fix bugs and I will write new features if I agree they are good for my users. All you have to do is ask!

Enjoy!

Update: I was so happy that Firefox for Android supports addons that I just installed it immediately and expected it to work. Unfortunately, the support for the Web Extensions API is very limited for the Android version, most importantly not having a bookmarks API, so the Bookmark Explorer doesn't work. I did make the extension more robust, though, by debugging it on the Android version.

I want to let you know about the latest features implemented in Bookmark Explorer.



The version number for the extension is already 2.9.3, quickly approaching the new rewrite I am planning for 3.0.0, yet every time I think I don't have anything else I could add, I find new ideas. It would be great if the users of the extension would give me more feedback about the features they use, don't use or want to have.

Here are some examples of new features:

• Skip button - moves the current page to the end of the bookmark folder and navigates to the next link. Useful for those long articles that you don't have the energy to read, but you want to.
• Custom URL comparison scheme. Useful for those sites where pages with different parameters or hash values are considered different and you get duplicate notification warnings for no good reason.
• Duplicate remover in the Manage page. This is an older feature, but now the button for it only appears where there are duplicates in the folder and with the custom URL scheme it's much more useful.
• Option to move selected bookmarks to start or end of folder, something that is cumbersome to do in the Chrome Bookmark Manager
• Automatically cleaning bookmark URLs of marketing parameters. This is in the Advanced settings section and must be enabled manually. So far it removes utm_*, wkey, wemail, _hsenc, _hsmi and hsCtaTracking, but I plan to remove much more, like those horrible hashes from Medium, for example. Please let me know of particular URL patterns you want to clean in your bookmarks and if perhaps you want the cleaning to be done automatically for all open URLs

As always, if you want to install the extension go to its Google Chrome extension page: Siderite's Bookmark Explorer

Just when I thought I don't have anything else to add, I found new stuff for my Chrome browser extension.
Bookmark Explorer now features:

• configurable interval for keeping a page open before bookmarking it for Read Later (so that all redirects and icons are loaded correctly)
• configurable interval after which deleted bookmarks are no longer remembered
• remembering deleted bookmarks no matter what deletes them
• more Read Later folders: configure their number and names
• redesigned options page
• more notifications on what is going on

The extension most resembles OneTab, in the sense that it is also designed to save you from opening a zillion tabs at the same time, but differs a lot by its ease of use, configurability and the absolute lack of any connection to outside servers: everything is stored in Chrome bookmarks and local storage.

Enjoy!

Bookmark Explorer, a Chrome browser extension that allows you to navigate inside bookmark folders on the same page, saving you from a deluge of browser tabs, has now reached version 2.4.0. I consider it stable, as I have no new features planned for it and the only changes I envision in the near future is switching to ECMAScript 6 and updating the unit test (in other words, nothing that concerns the user).

Let me remind you of its features:

• lets you go to the previous/next page in a bookmark folder, allowing sequential reading of selected news or research items
• has context menu, popup buttons and keyboard shortcut support
• shows a page with all the items in the current bookmark folder, allowing selection, deletion, importing/exporting of simple URL lists
• shows a page with all the bookmarks that were deleted, allowing restoring them, clearing them, etc.
• keyboard support for both pages
• notifies you if the current page has been bookmarked multiple times
• no communication with the Internet, it works just as well offline - assuming the links would work offline, like local files
• absolutely free

Install it from Google's Chrome Web store.

Update 17 June 2016: I've changed the focus of the extension to simply change the aspect of stories based on status, so that stories with content are highlighted over simple shares. I am currently working on another extension that is more adaptive, but it will be branded differently.

Update 27 May 2016: I've published the very early draft of the extension because it already does a cool thing: putting original content in the foreground and shrinking the reposts and photo uploads and feeling sharing and all that. You may find and install the extension here.

Have you ever wanted to decrease the spam in your Facebook page but couldn't do it in any way that would not make you miss important posts? I mean, even if you categorize all your contacts into good friends, close friends, relatives, acquaintances, then you unfollow the ones that really spam too much and you hide all posts that you don't like, you have no control over how Facebook decides to order what you see on the page. Worse than that, try to refresh repeatedly your Facebook page and see wildly oscillating results: posts appear, disappear, reorder themselves. It's a mess.

Well, true to this and my word I have started work on a Chrome extension to help me with this. My plan is pretty complicated, so before I publish the extension on the Chrome Webstore, like I did with my previous two efforts, I will publish this on GitHub while I am still working on it. So, depending on where I am, this might be alpha, beta or stable. At the moment of this writing - first commit - alpha is a pretty big word.

Here is the plan for the extension:

1. Detect the user has opened the Facebook page
2. Inject jQuery and extension code into the page
3. Detect any post as it appears on the page
4. Extract as many features as possible
5. Allow the user to create categories for posts
6. Allow the user to drag posts into categories or out of them
7. Use AI to determine the category a post most likely belongs to
8. Alternatively, let the user create their own filters, a la Outlook
9. Show a list of categories (as tabs, perhaps) and hide all posts under the respective categories

This way, one might skip the annoying posts, based on personal preferences, while still enjoying the interesting ones. At the time of this writing, the first draft, the extension only works on https://www.facebook.com, not on any subpages, it extracts the type of the post and sets a CSS class on it. It also injects a CSS which makes posts get dimmer and smaller based on category. Mouse over to get the normal size and opacity.

How to make it work for you:

1. In Chrome, go to Manage Extensions (chrome://extensions/)
2. Click on the Developer Mode checkbox
3. Click on the Load unpacked extension... button
4. Select a folder where you have downloaded the source of this extension
5. Open a new tab and load Facebook there
6. You should see the posts getting smaller and dimmer based on category.

Change statusProcessor.css to select your own preferences (you may hide posts altogether or change the background color, etc).

As usual, please let me know what you think and contribute with code and ideas.

I've written another Chrome extension that I consider in beta, but so far it works. Really ugly makeshift code, but I am not gathering data about the way I will use it, then I am going to refactor it, just as I did with Bookmark Explorer. You may find the code at GitHub and the extension at the Chrome webstore.

This is how it works: Every time you access anything with the browser, the extension will remember the IPs for any given host. It will hold a list of the IPs, in reverse order (last one first), that you can just copy and paste into your hosts file. The hosts file is found in c:/Windows/System32/drivers/etc/hosts and on Linux in /etc/hosts. Once you add a line in the format "IP host" in it, the computer will resolve the host with the provided IP. Every time there is a problem with DNS resolution, the extension will add the latest known IP into the hosts text. Since the extension doesn't have access to your hard drive, you need to edit the file yourself. The icon of DNS resolver will show the number of hosts that it wants to resolve locally or nothing, if everything is OK.

The extension allows manual selection of an IP for a host and forced inclusion or exclusion from the list of IP/host lines. Data can be erased (all at once for now) as well. The extension does not communicate with the outside, but it does store a list of all domains you visit, so it is a slight privacy risk - although if someone has access to the local store of a browser extension, it's already too late. There is also the possibility of the extension to replace the host with IP directly in the browser requests, but this only works for the browser and fails in case the host name is important, as in the case of multiple servers using the same IP, so I don't recommend using it.

There are two scenarios for which this extension is very useful:

• The DNS server fails for some reason or gives you a wrong IP
• Someone removed the IP address from DNS servers or replaced it with one of their own, like in the case of governments censorship

I have some ideas for the future:

• Sharing of working IP/host pairs - have to think of privacy before that, though
• Installing a local DNS server that can communicate locally with the extension, so no more hosts editing - have to research and create one
• Upvoting/Downvoting/flagging shared pairs - with all the horrible head-ache this comes with

As usual, let me know what you think here, or open issues on GitHub.

Update 29 August 2017 - Version 3.0.4: The extension has been rewritten in EcmaScript6 and tested on Chrome, Firefox and Opera.

Update 03 March 2017 - Version 2.9.3: added a function to remove marketing URLs from all created bookmarks. Enable it in the Advanced settings section. Please let me know of any particular parameters you need purged. So far it removes utm_*, wkey, wemail, _hsenc, _hsmi and hsCtaTracking.

Update 26 February 2017: Version (2.9.1): added customizing the URL comparison function. People can choose what makes pages different in general or for specific URL patterns
Update 13 June 2016: Stable version (2.5.0): added Settings page, Read Later functionality, undelete bookmarks page and much more.
Update 8 May 2016: Rewritten the extension from scratch, with unit testing.
Update 28 March 2016: The entire source code of the extension is now open sourced at GitHub.

Whenever I read my news, I open a bookmark folder containing my favorite news sites, Twitter, Facebook, etc. I then proceed to open new tabs for each link I find interesting, closing the originating links when I am done. Usually I get a number of 30-60 open tabs. This wreaks havoc on my memory and computer responsiveness. And it's really stupid, because I only need to read them one by one. In the end I've decided to fight my laziness and create my first browser extension to help me out.

The extension is published here: Siderite's Bookmark Explorer and what it does is check if the current page is found in any bookmark folder, then allow you to go forward or backwards inside that folder.

So this is my scenario on using it:

1. Open the sites that you want to get the links from.
2. Open new tabs for the articles you want to read or YouTube videos you want to watch,etc.
3. Bookmark all tabs into a folder.
4. Close all the tabs.
5. Navigate to the bookmark folder and open the first link.
6. Read the link, then press the Bookmark Navigator button and then the right arrow. (now added support for context menu and keyboard shortcuts)
7. If you went too far by mistake, press the left arrow to go back.

OK, let's talk about how I did it. In order to create your own Chrome browser extension you need to follow these steps:

1. Create the folder

Create a folder and put inside a file called manifest.json. It's possible structure is pretty complex, but let's start with what I used:

{
    "manifest_version" : 2,

    "name" : "Siderite's Bookmark Explorer",
    "description" : "Gives you a nice Next button to go to the next bookmark in the folder",
    "version" : "1.0.2",

    "permissions" : [
        "tabs",
        "activeTab",
        "bookmarks",
        "contextMenus"
    ],
    "browser_action" : {
        "default_icon" : "icon.png",
        "default_popup" : "popup.html"
    },
    "background" : {
        "scripts" : ["background.js"],
        "persistent" : false
    },
    "commands" : {
        "prevBookmark" : {
            "suggested_key" : {
                "default" : "Ctrl+Shift+K"
            },
            "description" : "Navigate to previous bookmark in the folder"
        },
        "nextBookmark" : {
            "suggested_key" : {
                "default" : "Ctrl+Shift+L"
            },
            "description" : "Navigate to next bookmark in the folder"
        }
    }
}

The manifest version must be 2. You need a name, a description and a version number. Start with something small, like 0.0.1, as you will want to increase the value as you make changes. The other thing is that mandatory is the permissions object, which tells the browser what Chrome APIs you intend to use. I've set there activeTab, because I want to know what the active tab is and what is its URL, tabs, because I might want to get the tab by id and then I don't get info like URL if I didn't specify this permission, bookmarks, because I want to access the bookmarks, and contextMenus, because I want to add items in the page context menu. More on permissions here.

Now, we need to know what the extension should behave like.

If you want to click on it and get a popup that does stuff, you need to specify the browser_action object, where you specify the icon that you want to have in the Chrome extensions bar and/or the popup page that you want to open. If you don't specify this, you get a default button that does nothing on click and presents the standard context menu on right click. You may only specify the icon, though. More on browserAction here.

If you want to have an extension that reacts to background events, monitors URL changes on the current page, responds to commands, then you need a background page. Here I specify that the page is a javascript, but you can add HTML and CSS and other stuff as well. More on background here.

Obviously, the files mentioned in the manifest must be created in the same folder.

The last item in the manifest is the commands object. For each command you need to define the id, the keyboard shortcut (only the 0..9 and A..Z are usable unfortunately) and a description. In order to respond to commands you need a background page as shown above.

2. Test the extension

Next you open a Chrome tab and go to chrome://extensions, click on the 'Developer mode' checkbox if it is not checked already and you get a Load unpacked extension button. Click it and point the following dialog to your folder and test that everything works OK.

3. Publish your extension

In order to publish your extension you need to have a Chrome Web Store account. Go to Chrome Web Store Developer Dashboard and create one. You will need to pay a one time 5$ fee to open it. I know, it kind of sucks, but I paid it and was done with it.

Next, you need to Add New Item, where you will be asked for a packed extension, which is nothing but the ZIP archive of all the files in your folder.

That's it.

Let's now discuss actual implementation details.

Adding functionality to popup elements

Getting the popup page elements is easy with vanilla Javascript, because we know we are building for only one browser: Chrome! So getting elements is done via document.getElementById(id), for example, and adding functionality is done via elem.addEventListener(event,handler,false);

One can use the elements as objects directly to set values that are related to those elements. For example my prev/next button functionality takes the URL from the button itself and changes the location of the current tab to that value. Code executed when the popup opens sets the 'url' property on the button object.

Just remember to do it when the popup has finished loading (with document.addEventListener('DOMContentLoaded', function () { /*here*/ }); )

Getting the currently active tab

All the Chrome APIs are asynchronous, so the code is:

chrome.tabs.query({
    'active' : true,
    'lastFocusedWindow' : true
}, function (tabs) {
    var tab = tabs[0];
    if (!tab) return;
    // do something with tab
});

More on chrome.tabs here.

Changing the URL of a tab

chrome.tabs.update(tab.id, {
  url : url
});

Changing the icon in the Chrome extensions bar

if (chrome.browserAction) chrome.browserAction.setIcon({
    path : {
        '19' : 'anotherIcon.png'
    },
    tabId : tab.id
});

The icons are 19x19 PNG files. browserAction may not be available, if not declared in the manifest.

Get bookmarks

Remember you need the bookmarks permission in order for this to work.

chrome.bookmarks.getTree(function (tree) {
  //do something with bookmarks
});

The tree is an array of items that have title and url or children. The first tree array item is the Bookmarks Bar, for example. More about bookmarks here.

Hooking to Chrome events

chrome.tabs.onUpdated.addListener(refresh);
chrome.tabs.onCreated.addListener(refresh);
chrome.tabs.onActivated.addListener(refresh);
chrome.tabs.onActiveChanged.addListener(refresh);
chrome.contextMenus.onClicked.addListener(function (info, tab) {
    navigate(info.menuItemId, tab);
});
chrome.commands.onCommand.addListener(function (command) {
    navigate(command, null);
});

In order to get extended info on the tab object received by tabs events, you need the tabs permission. For access to the contextMenus object you need the contextMenus permission.

Warning: if you install your extension from the store and you disable it so you can test your unpacked extension, you will notice that keyboard commands do not work. Seems to be a bug in Chrome. The solution is to remove your extension completely so that the other version can hook into the keyboard shortcuts.

Creating, detecting and removing menu items

To create a menu item is very simple:

chrome.contextMenus.create({
  "id" : "menuItemId",
  "title" : "Menu item description",
  "contexts" : ["page"] //where the menuItem will be available
});

However, there is no way to 'get' a menu item and if you try to blindly remove a menu item with .remove(id) it will throw an exception. My solution was to use an object to store when I created and when I destroyed the menu items so I can safely call .remove().

To hook to the context menu events, use chrome.contextMenus.onClicked.addListener(function (info, tab) { }); where info contains the menuItemId property that is the same as the id used when creating the item.

Again, to access the context menu API, you need the contextMenus permission. More about context menus here.

Commands

You use commands basically to define keyboard shortcuts. You define them in your manifest and then you hook to the event with chrome.commands.onCommand.addListener(function (command) { });, where command is a string containing the key of the command.

Only modifiers, letters and digits can be used. Amazingly, you don't need permissions for using this API, but since commands are defined in the manifest, it would be superfluous, I guess.

That's it for what I wanted to discuss here. Any questions, bug reports, feature requests... use the comments in the post.

I met this situation where I wanted to implement an http interceptor, a piece of JavaScript code that would do something on unauthorized access to my API. The way to do this is irrelevant (and different from jQuery and AngularJS sites), but there is a problem that affects every situation and that is when you access the API from a different domain than the API's. You see, the browser needs the API server to authorize CORS for every Ajax request that accesses that server from another domain. You might think you did that already in your API, but let me ask you: when there is a problem, like not authorized access, are you sending CORS headers with your error response? Because if you do not, everything you send, including the http code, will not be parsed by the browser and any interception will just show a code of 0. The situation is a little confounded by the fact that the browser does announce that you have a CORS access problem, but also displays the status message, which in this case would be "Unauthorized access". This might make you think you can access that message or the status code. Well, you cannot.

The solution is to send the CORS headers: Access-Control-Allow-Origin, Access-Control-Allow-Methods, Access-Control-Allow-Headers. Just set their value to "*" and send them with your error response and now your JavaScript can read the error code and message.

Sometimes you want to run your browser without some protections that are enabled by default in it. One of these is the cross-origins protection when running a local filesystem script. For Chrome, the solution is to run the browser with the command line switch --allow-file-access-from-files. Seems straight forward enough, but if your browser is already open (usually I have at least 10 active tabs, ranging from documentation pages, email to the music I listen to), the command line switches will be ignored and your script will be run as just another window in the same instance of Chrome. In order to fix this, you need to use another switch called --user-data-dir. Just make sure this folder exists and it can be deleted (because it will be filled with a zillion useless files).

So, how do I run a test.html file that can access files and is located in C:\SomePath ? Like this:
start "Chrome with access to files" chrome --allow-file-access-from-files "file:///C:\SomePath\test.html" --user-data-dir="C:\SomePath\UserDir"

In your path a UserDir folder will be created which you can delete after you finish your work.

Of course, this issue applies to any other flags that you want to use ad-hoc while Chrome is already open.

Update 12 Feb 2016: As an alternative, if you have access to your hosts file, you can use a generated list of domains to immediately block any access to those IPs, courtesy of Steven Black: StevenBlack/hosts, and you don't have to install anything and works on most operating systems.

Update 19 Feb 2016: Here are the latest files to directly download and use. Do read the rest of the article, as these files might not be what you are looking for:
user.action
user.filter

Now for the rest of the article:

I discovered today a new tool called Privoxy. It is a proxy software that has extra features, like ad blocking and extra privacy. What that means is that you can install the proxy, point your browser to that proxy and have an almost ad free untracked by marketing firms or FaceBook experience. The only problem, though, is that the default filters are not so comprehensive. Would it be great if one could take the most used list for ad blocking (Adblock Plus' Easylist) and convert it to Privoxy? Well it would, only that no one seems to want to do it for Windows. You get a few folks that have created Linux scripts to do the conversion, but they won't work for Windows. Nor do they deem it necessary to make an online tool or a web service or at least publish the resulting files so that we, Windows people, can also use the list!

Well, I intend to do a small script that will allow for this, preferably embedded in this blog post, but until then, I had no script, no files, only Privoxy installed. Luckily, I also have Cygwin installed, which allows me to run a ridiculous flavour of Linux inside Windows. I had to hack the script in order for it to work on Cygwin, but in the end, at last, I managed to make it work. For now, I will publish the resulting files here and give you instructions to install them. When they become obsolete, send me a comment and I will refresh them.

So, the instructions:

• Install Privoxy
• Go to the installation folder of Privoxy and look for files named 'user.action' and 'user.filter'
• Download the user.action file from here and replace the default one.
• Download the user.filter file from here and replace the default one.
• Restart Privoxy
• Of course, then you have to go to your browser settings and set the proxy to Privoxy (usually localhost, port 8118)

Warning! The filter file is quite big and it will cause some performance issues. You might want to use only the action file with the filter actions removed.

Update: If you can't download the files, let me know. I am using Github pages and it seems sometimes it doesn't work as I expected.

Also, I have lost faith that AdBlockPlus rules can be completely and correctly translated to Privoxy and I lack the time, so I am publishing my crappy program as it is and if you want to use it or fix it, be my guest. You can find the program here: AdblockPlus to Privoxy. I would ask that you share with me those modifications, though, so that everybody can benefit from them.

Update October 2014: Other people have contributed by making their own translation software. Here are the links for the binary and source code of adblock2privoxy made by Zubr, in Haskell mind you, which is pretty cool:

Binary: https://www.dropbox.com/s/69u1iqbubzft1yl/adblock2privoxy-1.2.4.rar?dl=0
Source code: https://projects.zubr.me/wiki/adblock2privoxy