CKAN is an open source management system whose main goal is to provide a managed data-catalog-system for Open Data. It is mainly used by public institutions and governments. At Liip we use CKAN to mainly help governments to provide their data-catalog and publish data in an accessible fashion to the public. Part of our work is supporting data owners to get their data published in the required data-format. We’re doing this by providing interfaces and useable standards to enhance the user experience on the portal to make it easier to access, read and process the data.

Metadata-Catalog

Out of the box CKAN can be used to publish and manage different types of datasets. They can be clustered by organizations and topics. Each dataset can contain resources which themself consist of Files of different formats or links to other Data-Sources. The metadata-standard can be configured to represent the standard you need but the Plugin already includes a simple and useful Meta-Data-Standard that already can get you started. The data is saved into a Postgres-Database by default and is indexed using SOLR.

Powerful Action-API

CKAN ships with an API which can be used to browse through the metadata-catalog and create advanced queries on the metadata. With authorization the API can also be used to add, import and update data with straight-forward requests.

Cli-Commands

The standard also includes a range of Cli-Commands which can be used to process or execute different tasks. Those can be very useful, e.g. to manage, automate or schedule backend-jobs.

Preview

CKAN offers the functionality to configure a preview of a number of different file-types, such as tabular-data (e.g. CSV, XLS), Text-Data (e.g. TXT), Images or PDFs. That way interested citizens can get a quick overview into the data itself without having to download it first and having to use local Software to merely get an better idea on how the data looks.

Plugins

While CKAN itself acts as a CMS but for data, it really shines when making use of its extensibility and configure and develop it to your business needs and requirements. There is already a wide-ranging list of plugins that have been developed for CKAN, which covers a broad range of additional features or make it easier to adjust CKAN to fit your use cases and look and feel. A collection of most of the plugins can be found on CKAN-Extensions and on Github.

At Liip we also help maintaining a couple of CKAN's plugins. The most important ones that we use in production for our customers are:

ckanext-harvest

The ckanext-harvest-plugin offers the possibility to export and import data. First of all, it enables you to exchange data between Portals that both use CKAN.

Furthermore we use this plugin to harvest data in a regular manner from different data-sources. At opendata.swiss we use two different types of harvesters. Our DCAT-Harvester consumes XML-/RDF-endpoints in DCAT-AP Switzerland-Format which is enforced on the Swiss Portal.

The Geocat-Harvester consumes data from geocat.ch. As the data from geocat is in ISO-19139_che-Format (Swiss version of ISO-19139) the harvester converts the data to the DCAT-AP Switzerland format and imports it.

Another feature of this plugin we use, is our DCAT-AP endpoint, to allow other portals to harvest our data and also serves as an example to Organizations that want to build an export that can be harvested by us.

ckanext-datastore

The plugin ckanext-datastore stores the actual tabular data (opposing to 'just' the meta-data) in a seperate database. With it, we are able to offer an easy to use API on top of the CKAN-Standard-API to query the data and process it further. It provides basic functionalities on the resource-detail-page to display the data in simple graphs.

The datastore is the most interesting one for Data-Analysts, who want to build apps based on the data, or analyze the data on a deeper level. This is an API-example of the Freibäder-dataset on the portal of Statistik Stadt Zürich.

ckanext-showcase

We use ckanext-showcase to provide a platform for Data-Analysts by displaying what has been built, based on the data the portal is offering. There you can find a good overview on how the data can be viewed in meaningful ways as statistics or used as sources in narrated videos or even in apps for an easier everyday life. For example you can browse through the Showcases on the Portal of the City of Zurich.

ckanext-xloader

The ckanext-xloader is a fairly new plugin which we were able to adopt for the City of Zurich Portal. It enables us to automatically and asynchronously load data into the datastore to have the data available after it has been harvested.

CKAN Community

The CKAN-Core and also a number of its major plugins are maintained by the CKAN-Core-Team. The developers are spread around the globe, working partly in companies that run their own open-data portals. The community that contribute to CKAN and its Plugins is always open to developers that would like to help with suggestions, report issues or provide Pull-Requests on Github. It offers a strong community which helps beginners, no matter their background. The ckan-dev-Mailing-List provides help in developing CKAN and is the platform for discussions and ideas about CKAN, too.

Roadmap and most recent Features

Since the Major-Release 2.7 CKAN requires Redis to use a new system of asynchronous background jobs. This helps CKAN to be more performant and reliable. Just a few weeks ago the new Major-Release 2.8 was released. A lot of work on this release went into driving CKAN forward by updating to a newer Version of Bootstrap and also deprecating old features that were holding back CKAN's progress.

Another rather new feature is the datatables-feature for tabular data. Its intention is to help the data-owner to describe the actual data in more detail by describing the values and how they gathered or calculated.

In the Roadmap of CKAN are many interesting features ahead. One example is the development of the CKAN Data Explorer which is a base component of CKAN. It allows to converge data from any dataset in the DataStore of a CKAN instance to analyze it.

Conclusion

It is important to us to support the Open Data Movement as we see value in publishing governmental data to the public. CKAN helps us to support this cause by working with several Organizations to publish their data and consult our customers while we develop and improve their portals together.

Personally, I am happy to be a part of the CKAN-Community which has always been very helpful and supportive. The cause to help different Organizations to make their data public to the people and the respectful CKAN-Community make it a lot of fun to contribute to the code and also the community.

After each optimization I thought that I understood how the srcset and sizes attributes of the <img> tag works. But each time, I made some new findings about how these attributes really work and where I need to take care while implementing them. Since there are lots of things you should know before using srcset I thought it could be useful to summarize the most tricky parts in a blog post.

A short introduction

I think most of you already heard of responsive images and especially the srcset attribute. But to be on the same page I still would like to start with a short introduction about these two attributes:

srcset

The srcset attribute is listing different resolutions of the same image from which the browser chooses the best fitting image source before loading it.

Example: srcset="ninja-1000w.jpg 1000w, ninja-500w.jpg 500w, ..."

To calculate this, the browser assumes the image fills up the full viewport width (100vw) by default, which means it uses the full width of the browser.

sizes

To tell the browser how much space the image really needs on our viewport, we can use the sizes attribute. This attribute contains a comma separated list of one or more image widths for different viewport sizes.

Each entry is a combination of a media condition and a width. Both of these values are described in CSS pixels so we don't need to care about device pixel ratio for this. If the media condition is omitted it evaluates to true automatically (= fallback). The sizes attribute gets read from left to right. As soon as a media condition evaluates to true the width of this entry is used. So be sure to order your values correctly!

Example: sizes="(min-width: 1000px) 50vw, 100vw"

The example above tells the browser that if the viewport is at least 1000px wide the image fills 50% of the space. If the browser is smaller, the image uses 100% of the available width.

If we combine the above two examples in an <img> tag and request it with a (1x/non-retina) browser the result would be the following:

• Browser width: 500px -> Matching sizes width: 100vw -> Needed image size: 500w -> Chosen image: ninja-500w.jpg.
• Browser width: 900px -> Matching sizes width: 100vw -> Needed image size: 900w -> Chosen image: ninja-1000w.jpg.
• Browser width: 1000px -> Matching sizes width: 50vw -> Needed image size: 500w -> Chosen image: ninja-500w.jpg.

Sizes affect how the image is shown

The first thing which is important to know about the sizes attribute is that it isn't only used to calculate the needed size, it is used as the rendered width of the image if the width is not defined in CSS too.

This means as soon as you add the srcset attribute to the <img> element the image might be displayed differently.

The reason for that is when you add the srcset attribute and omit the sizes attribute, the browser uses the default value for it, which is 100vw.

Here's an example (https://codepen.io/tschortsch/pen/LeMmyO):

As you can see, the browser stretches the image always to the matching size in the sizes attribute, as long as the width is not defined in CSS.

This leads us to Rule #1: Always set the sizes attribute if you use srcset. If sizes is omitted it defaults to 100vw.

How the browser selects the needed image size

Short answer: We can't tell.

Long answer: Every browser has a slightly different implementation which can change with every version of the browser. This leads to Rule #2: Never assume which image size the browser will choose.

The browser just chooses the best matching image for the current size. But if the browser finds a cached version of an image from the current srcset which is bigger than the needed size it prioritizes the image from the cache for example.

This makes it really hard to debug a srcset. To avoid the loading of cached files you should always debug srcsets in your browsers privacy mode.

As a result of this, it is possible that you have two exact same devices (same screen and browser size) but you get a different image size in both browsers.

Another really important takeaway from this brings us to Rule #3: A srcset should only contain images of the same ratio.

Since you can't really decide which image is loaded in which browser size you can't use srcset to serve different images on different sizes a.k.a. art direction (eg. 1:1 image on smaller devices, 2:1 image on larger devices). If you would like to do this you should use the <picture> element.

Pitfalls of the width ('w') descriptor

The width (w) descriptor of the srcset attribute has also some things which should be taken care of. First of all Rule #4: The width (w) descriptor should always match the images natural width. This means if the image has a width of 500px the width (w) descriptor should be exactly 500w. This sounds pretty easy to achieve but there are use cases where this isn't as easy.

Think of the following example: You load your images through a CDN, where you predefine all the sizes that should exist of an uploaded image. In the template you define a srcset with all of those predefined sizes. If you do not enable upscaling (on CDN side) and upload an image which is smaller than the biggest defined size, you will get an image which doesn't fit the width (w) descriptor in your template.

If this is the case you can get unexpected results. Let's look at this example (https://codepen.io/tschortsch/pen/rpoXvW / The problem in the example only occurs with screens that have a DPR >= 2):

If you don't have a display that has a DPR >= 2 here's a screenshot of what would happen if you had:

What is showed in the example above? As long as we don't define a CSS width, the image (which is originally 400px wide) doesn't fill up a container which is only 300px wide.

Let me explain what's happening here: We have an image container which is 300px wide. The image has a srcset with two possible image sizes: 300w, 600w. The 300w image has a natural width of 300px (correct). But since the original image has a width of 400px the 600w doesn't return an image which is 600px wide but the original 400px image (wrong).

If this container gets loaded with a DPR >= 2 the browser requests the 600w image (2 * 300px). Since the browser doesn't know that the image behind this descriptor is smaller than 600px it displays it as it would be that size. This means it gets squeezed to half of its size (400px / 2 = 200px). And that's why we end up with this strange situation that the image (which is originally 400px wide) doesn't fill the whole 300px image container.

When we enforce the image width in CSS, like we do in the 2nd example, the (400px) image gets stretched again to fill the 300px container and everything looks like expected.

And here we are with the Rule #5: Always enforce the image size in CSS when using srcset.

Wrap up

As you see responsive images have some parts, you should know when implementing them. Let's quickly go through our rules again:

• Rule #1: Always set the sizes attribute if you use srcset. If sizes is omitted it defaults to 100vw. Since the sizes attribute has an influence on how the image is displayed, it should never be omitted. If you do so, the browser uses the default value of 100vw.
• Rule #2: Never assume which image size the browser will choose. The reason for this is, that all browsers have slightly different implementation on how the correct image from a srcset is chosen. This implementation should be a black box for us, because it can change with every update.
• Rule #3: A srcset should only contain images of the same ratio. As a result of rule #2 we never know which image is loaded by the browser. To get the same result for all visitors every image in the srcset needs the same ratio.
• Rule #4: The width (w) descriptor should always match the images real width. The browser uses the width (w) descriptor to calculate how it should display the image. If it doesn't match the real width, we can get unexpected behaviour.
• Rule #5: Always enforce the image size in CSS when using srcset. Since there are some use cases where rule #4 can't be achieved, we should always enforce the width of an image in CSS.

Further resources

There are some really good resources to learn more about responsive images:

• Responsive images on Mozilla Developer Network: https://developer.mozilla.org/en-US/docs/Learn/HTML/Multimedia_and_embedding/Responsive_images
• Responsive images on CSS-Tricks: https://css-tricks.com/responsive-images-youre-just-changing-resolutions-use-srcset/
• Very nicely done introduction about srcset and sizes: http://ericportis.com/posts/2014/srcset-sizes/
• <img> Element documentation: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/img

What is OctoberCMS?

WordPress is based on simplicity - everything is a post, which makes it easy to use for content authors and developers. Thanks to this concept, (nearly) all backend forms look consistent which helps content authors a lot. But this simplicity makes it harder for developers to create backends for more complex content.

This is where OctoberCMS jumps in. It is a great mix of a file based and database driven CMS. Everything which is more or less static (like pages, menus and templates) is saved as files. Like this the content can be served really fast. However you can of course edit these entities directly through the backend.

Everything which is more dynamic and more complex (like events, news or blogposts) is stored in custom tables in the database. Thanks to this you will have full freedom on how to create your own types.

Complexity doesn't matter

With this strategy OctoberCMS can be used for simple websites as well as for really complex ones. When using it for simple websites, the user experience of the backend can be compared to the one of WordPress. It is easy to use and looks consistent. On top of that it adds some useful features for power users, like editing multiple content elements in tabs or keyboard shortcut. The most useful one is saving data with CMD+S / CTRL+S. I'm already at a point where I find myself trying to save blogposts in WordPress with this key combination (which doesn't work sadly).

When using it for a more complex website you'll get full freedom on what to use OctoberCMS for. You can easily use it as a headless CMS for any sort of web application. To help you with this, the backend gives you a base structure for list views and form views for each model. If required you can modify these templates to fit your own needs. The provided form widgets helps creating consistent backend forms for almost every use case. If needed you can create your own form widgets without any hassle. This means the backend is highly customizable which improves the usability when editing complex content extremely.

Still not convinced?

Let's look at some more, great reasons why you should consider OctoberCMS for your next website:

• Based on Laravel: OctoberCMS is based on the Laravel framework (which is based on Symfony. It builds a really solid foundation for the core, plugins and themes.
• Clean code: By using the Laravel framework there is clear coding style definition which helps your code base to stay clean.
• Open Source: The whole OctoberCMS project is hosted as open source software on GitHub which is a really important thing to build a big user community around.
• Easy to extend: OctoberCMS is built for extensibility. The best example for that is that even the most basic features, like creating and editing pages, are provided via a plugin and are not hardcoded in the core.
• Database query building with Eloquent: For database query building Laravel has Eloquent ORM integrated. With this you can easily create database queries which fits your needs.
• Database migrations: The whole OctoberCMS setup (core and plugins) is based on database migrations. This helps keeping the database consistent when installing or updating plugins or the core. With this mechanism you can easily rollback to an older version if something doesn't work as expected.
• Event driven: The whole setup is event driven which enables you hook into core or plugin processes and extend them easily.
• Twig templates: OctoberCMS uses twig as templating engine. This makes it possible to completely separate your data from the templates.
• Integrated assets pipeline: OctoberCMS comes with a twig based assets pipeline to compile and minify your CSS (support for Sass & Less) and JS files. All you need to do is including your source files directly in your twig templates and they get compiled, minified and cached automatically when calling the website the first time. If you have more complex requirements Laravel provides an abstraction layer above webpack called Laravel Mix which makes the creating of webpack based build tasks easy.

The bad parts

Not everything can be nice and shiny. So here we go with the imperfect or missing parts of OctoberCMS:

Core bugs

First of all since this is a rather new CMS project (compared to other big players on the market) you might find some more or less "simple" core bugs while implementing a complex backend feature. This can be annoying, but since the community is very active (on Slack or on GitHub) you will get help quickly.

i18n

Another important topic is i18n: OctoberCMS provides an implementation for translating content as a plugin. With it you are able to translate most of your content from a base language to other languages which works fine.

But since you always need your content to exist in a base language you are limited to only translating your website and not doing real i18n stuff like different content in different languages. The good thing about this is that the folks behind this plugin are working on that feature right now and since the plugin is hosted on GitHub you can help if you want.

That's all folks

We're really looking forward to build a lot more websites with OctoberCMS. And if you are a web developer you should really consider it when planning your next CMS project.

The main feature from my point of view is that with SPDY the browser only has to open one TCP connection to the server and all resources are delivered through that connection. No need anymore to “fake” different servers with different hostnames just that the browser fetches more than the default 4 to 6 resources at once from one server. This also saves resources on the server as it also has only to keep one connection open. This especially helps, if you have lots of little resources on your webpage, eg. lots of small images.

SPDY also adds features that a lost package doesn't stop the whole delivery and other requests still can get through fast called multi-plexing (compared to HTTP Pipelining, which is First-In-First-Out) and since everything is going over one TCP connection, SPDY can much better handle congestion and better use the available bandwidth (see also this post for more info)

Furthermore it also compresses the HTTP Headers (something HTTP doesn't by design and can sum up to quite some traffic, especially if you have many small requests)

You don't have to change anything in your websites and web applications to make use of SPDY, only the browser and the webserver have to support it, on the other layers it still works like HTTP.

All this together should especially help on mobile devices, where latency, packet loss and bandwidth still is a problem and any improvements in that area helps a lot in making a more responsive web experience.

SPDY on this blog

There was actually no real need for having SPDY on our blog. We don't have that many requests on a single page to the same server and our server isn't really overloaded with open TCP connection (currently it's an AWS micro instance, not the most powerfull machine in the world ;)). But it is a little playground for us to try out new things, get some experience with those and to be able to make an informed decision if we should/could use them in our projects. So I went ahead.

Google provides packages for Apache SPDY modules, so that was installed fast. But our server was running mod_php5 and that doesn't work very well with mod_spdy (which is somehow obvious, since SPDY just opens one TCP connection for all (parallel) requests, something hard to do with the pre-fork model mod_php5 needs). So I had to move my PHP needs to PHP-FPM and use FastCGI and the Worker MPM of apache. There wasn't anything special I had to do. It's the usual setup of using PHP-FPM with apache (see for example here) and since the mod_rewrite rules still work, I didn't even have to change them. After that and installing mod_spdy_beta.deb I basically had a running SPDY site on our SSL port.

This is also one of the issues and main criticisms  about SPDY. It only runs with SSL (currently). That has the advantage that the future HTTP may be secure by default, but also the disadvantage that you need certificates and all that to set it up. This wasn't difficult for this site since we already had SSL running for the admin interface. The next question was how to bring visitors to the SPDY version of the blog. Just send everyone to SSL didn't sound like a good idea, because with “old HTTP” SSL has more overhead in setting up a connection and since you will have many of those connections, that really will be a penalty. One trick is to send the following header to all clients connecting on port 80

Alternate-Protocol: 443:npn-spdy/2

Then SPDY-enabled clients will use SPDY for the upcoming requests. This works fine on Chrome, on Firefox I had the problem that our webfont somehow wasn't loaded and displayed. So I added this to our mod_rewrite rules:

RewriteCond  %{HTTP_USER_AGENT}  Firefox/[0-9]{2}.
RewriteCond  %{SERVER_PORT}  ^80$
RewriteRule ^/*(.*)$ https://blog.liip.ch/$1 [L,R=301]

This redirects all Firefox versions >= 10. to SSL. Not the best solution, since the redirect adds latency, but for the sake of it, it's good enough for me here. I will revisit the webfont issue later, maybe it is indeed a bug in Firefox which will be fixed later.

That was basically it, if you now go to our blog with Chrome or Firefox, you should be served by SPDY. You can check that by installing one of the SPDY indicator plugins ( Chrome, Firefox) or in Chrome go to  chrome://net-internals/#spdy and see which sites you have open use SPDY (twitter and google are 2 major sites already using SPDY).

One thing you have to also keep an eye on is that once you deliver your site in SSL, you should deliver all content over SSL. Otherwise you may get warnings in the browser (I didn't change all the externals URLs in the individual posts on this blog, so you may hit that warning from time to time)

What about nginx?

This all could be done also with nginx (apart from our very customized mod_rewrite rules which prevent a totally easy move), but SPDY support was only published last week and you have to patch and recompile nginx (no binaries yet). Not something I was very keen to do. But support is coming and certainly a standard in the near future.

Outlook

I really hope SPDY (or something similar) will gain widespread adoption in the near future. Everything which makes delivering web resources faster, especially on high-latency, packet-lossy and limited-bandwidth environments is worth pursuing and it's obvious that SPDY does solve some problems in that area (if you ever tried to surf the web on a full Swiss train you know what I'm talking about ;))

And with the compiled packages by google it's easy to set it up on your server, provided you use apache, some fastcgi for PHP and have a decent SSL certificate already (many ifs…). If that's the case, why not playing with it and gain some experience.

I also can't wait the day when varnish supports SPDY natively. There are no plans as far as I know and it doesn't look like they're willing to do it soon. But if it gains more widespread adoption, I'm sure they will support it some day. Until then your only way to have SPDY and varnish is to put an nginx/apache in front of varnish. Which solves the problems mentioned above for mobile networks, but not the “too many open connections on the server side” problem. But that's mitigated somehow anyway if you use varnish.

And last but not least, for me as an Apple follower, I can't wait until it's built-in in iOS ;)

Now that all the videos of the Fronteers talks are online, I wanted to shared with you what I came back with, so that you also go benefit from this resource.

The Venue

I have been to some conferences, but this venue just rocked them all. We were hosted in the Pathé Tuschinski movie theater in the middle of Amsterdam. The pictures I took of this place describe how perfect this place was, much better than I could explain it with words:

The Lineup

The lineup of speakers was pretty impressive. Paul Irish, Jeremy Keith, Nicholas Zakas, Robert Nyman, Christian Heilmann, Stoyan Stefanov to name some of them. A whole bunch of wise guys. It was a blessing to listen to what they had to share.

The topics

CSS3 , javascript and of course everything around HTML5 were the trending topics. Lately, I somehow got an overdose of hearing about HTML5 everywhere again. But those speakers all presented their stuff with their own perspective and that made it really pleasant and inspiring to follow.

Jeremy Keith, for example, took us through the design principles on which HTML5 is built. I am still fascinated on the vision behind this standard. I truly believe such principles are what made the current standard a success and what will ensure its future.

Håkon Wium Lie, CTO of opera and long time member of the CSS working group went with us through the new features of CSS3 . He shared some pictures and anecdotes of the times he spent with Tim Berners Lee in the CERN, about 20 years ago, to illustrate what was inspiring them while creating the concept of CSS.

Meagan Fisher and Jina Bolton gave us their web designer perspective. Meagan showed us how she uses the new CSS3 properties to give even a better look to her designs on browsers that support them. Jina told us about the workflow she goes through while writing CSS.

I'm not a designer, but I envy their talent and creativity. A journey in a mind of such designers was a real gift to me.

I think you got the point and I won't comment every talk as you can find them online. Let me just recommend three of them to start with.

The 3 talks you should see

“ Progressive Downloads and Rendering” by Stoyan Stefanov

If you care at lot about performance for high traffic websites this is the talk to watch. Probably the talk where I learnt the most.

“ Reusable Code, for good or for awesome!” by Jake Archibald

Jake talked about how to design an API and making sure it's reusable. He mostly uses javascript as example, but the principles he presents (and which I cherish too) are relevant to any coding environment. The bonus of this talk is how entertaining that guy is.

“ Reasons to be cheerful” by Chris Heilmann

This is THE talk I would recommend to everyone working for the web. For about an hour he tells us why we have one of the most awesome jobs in the world. I enjoyed sitting there and contemplating his speaker talent delivering all the reasons why I do this job.