OS Zoomstack Vector Tile Set
Creating a vector tile set of OS Zoomstack data and consuming it in QGIS
For this post I wanted to investigate using the new Zoomstack data set by Ordnance Survey(OS). This is a great new product that has been released in two main data types, Geopackage and MBtiles. Both are relatively new data types that the OS is utilizing well in their attempt to modernise its Open Data offerings.
Both use SQLlite to store vector data. MBtiles is a new standard created by Mapbox that is way of collecting tile sets into one file. It allows for multiple zoom levels of data from a Tile set to be stored in one SQLLite file and can store multiple layers and data types, it can be raster or vector.
I was investigating a way of using Zoomstack for our companies needs when I came across an article by James Milner.
I found this article very useful and it showed me a simple way to serve up tiles. However, I thought a more detailed post with a step-by-step guide might be useful to people new to tile sets, tile servers or even just OS data.
I had thought about using Tippecanoe, an open library created by Mapbox to convert GeoJson and other vector formats to raw vector tiles— tile sets. But James Milner’s post showed me a different part of the library I hadn’t seen before that of tile-join. The main premise of Tippecanoe is to allow for detailed mapping at all zoom levels rather than relying on cartographic generalization tools such as simplification and selection. Generalization was used to primarily make a map more readable by emphasising key components, but it was also used in the early web mapping era to render less information thus creating faster more efficient mapping services. However, with modern vector tile services and faster servers, it is now possible to serve up fast efficient maps that retain all or most of the data regardless of zoom level.
From a cartographic point of view, I would personally not want this feature in a map service created solely for background mapping that will have other data sets overlain. This is because of detail competition, the details within the background mapping could compete with the main data sets that are overlain on the map. Tippecanoe does have some toggles that allow for generalization of data both simplification of polygons, selection of points etc. See their GitHub page for more details.
For this post I’m not going to go into more detail about Tippecanoe and cartographic generalization, but it might be a good subject to deep dive into for another post. In this instance we are using tile-join function of the library to create the tile set, Tippecanoe will do the work for us to extract the data from the MBTile and create individual vector PBF tiles in the correct folder structure.
To begin we are going to create a simple vector service from OS open data that will give us a simple background mapping solution for the UK. We will then use QGIS to consume the service, but any GIS software or web mapping APIs that can consume tile services will work, however I will note that some mapping tools do not work with compressed tiles.
The first step is to download the MBTile format of the Zoomstack data set from OS Open Data portal. To install Tippecanoe will depend on your environment, but I am using Ubuntu within Windows — WSL. On Mac, it is pretty easy, just use;
brew install tippecanoeFor Linux or WSL the process is slightly more difficult but follow the development guide. Once installed type tile-join into the console to check that the library has been installed correctly. You should see something like the following in your console;
The command to convert the MBTiles to a pbf tile set is quite simple,
tile-join -—output-to-directory=tiles OS_Open_Zoomstack.mbtilesWith the above command all zoom levels will be copied to the vector tile set with compression applied. Depending on your user case you might not want to use compression, for instance Mapbox-GL-JS does not read compressed tiles out of the box. In this instance QGIS can read and render compressed tiles fine, however it would be interesting to see if there is a render speed penalty with compressed tiles, as the rendering is done user side.
Run the above command and now wait, it could take quite a while. You should start to see folder structures being created. While waiting for this command to run, this could be a good time to read up on tile services and how they work. The basics is that the tiles are placed into folder structures that represent z,x,y of the tile. Tile sets are a way of dividing an area into regularly sized tiles that split the map bounds into smaller and smaller squares the higher the zoom level. Therefore, our finished tile set is a series of folders and pbf vector files that hold the actual data.
Once the tile-join command finishes, you should now have 14 folders with sub folders holding our tile set. Now we need to create a http-server that will serve up each tile on request. Make sure you have Node installed and use the following to install http-server;
npm install -g http-serverThen run the below code to start the service;
http-server –corsThe above command will start a http service, make sure you start the service in the parent folder where the tiles are stored, in my case;
/c/work/zoomstack-serviceIf you ran the same command above you should have a sub folder called tiles that holds our new tile-set;
Now go to the URL as shown in the console and you should see a list of folders and files. To test our service is working we can ask the new server for a tile with the following;
http://YourIPHere:8080/tiles/0/0/0.pbfThis should download a vector tile, confirming that our service is up and running.
Next, we will use the Vector-Tiles-Reader plugin for QGIS to view the data. The plugin has a useful readme on how to configure the service connections.
First up, we need to create a config json file that the plugin can read to open our new service. Let’s create a config folder in the same directory that the tiles directory is in. Our new service will also serve up the config file for Qgis to read. Copy the below config file and save it as config.json within our new config folder.
This config file will tell Qgis which layers to open from the Zoomstack tilesets, but it won't add any automatic styles to our data (this is something I might explore in another post), for now we can just view the data. Open the vector tiles reader plugin in Qgis and in the connections section add a new connection, name the connection and for TileJson Url connect to the config file from your service:
http://YourIPHere:8080/config/config.jsonAfter clicking next, you should get something like below,
Click Base Map Defaults that will add random styling to our new service and click Add. And there you have it; you should now have vector tiles being rendered from the Zoomstack service. In another post I will investigate other ways of consuming vector tiles in QGIS and Mapbox-GL-JS. As well as setting this service up on an AWS EC2 instance.
