cerxes/openmaptiles
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

merge master into add-garages

Browse Source
This commit is contained in:
golubev 2019-12-06 19:40:12 +02:00
commit 1247972951
23 changed files with 228 additions and 175 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
.travis.yml
View File

@ -1,22 +1,9 @@
sudo: required
dist: trusty
language: bash
services:
- docker
before_install:
- sudo rm -f /usr/local/bin/docker-compose
- curl -L https://github.com/docker/compose/releases/download/1.7.1/docker-compose-`uname -s`-`uname -m` > docker-compose
- chmod +x docker-compose
- sudo mv docker-compose /usr/local/bin
- sudo service docker restart
- docker-compose -v
- docker -v
script:
- sudo service docker restart
- docker-compose config
- sudo ./quickstart.sh
- ./quickstart.sh

20
Makefile
View File

@ -74,7 +74,7 @@ download-geofabrik:
@echo " "
psql: db-start
docker-compose run --rm import-osm /usr/src/app/psql.sh
docker-compose run --rm import-osm ./psql.sh
import-osm: db-start all
docker-compose run --rm import-osm
@ -191,29 +191,29 @@ download-wikidata:
mkdir -p wikidata && docker-compose run --rm --entrypoint /usr/src/app/download-gz.sh import-wikidata
psql-list-tables:
docker-compose run --rm import-osm /usr/src/app/psql.sh -P pager=off -c "\d+"
docker-compose run --rm import-osm ./psql.sh -P pager=off -c "\d+"
psql-pg-stat-reset:
docker-compose run --rm import-osm /usr/src/app/psql.sh -P pager=off -c 'SELECT pg_stat_statements_reset();'
docker-compose run --rm import-osm ./psql.sh -P pager=off -c 'SELECT pg_stat_statements_reset();'
forced-clean-sql:
docker-compose run --rm import-osm /usr/src/app/psql.sh -c "DROP SCHEMA IF EXISTS public CASCADE ; CREATE SCHEMA IF NOT EXISTS public; "
docker-compose run --rm import-osm /usr/src/app/psql.sh -c "CREATE EXTENSION hstore; CREATE EXTENSION postgis; CREATE EXTENSION unaccent; CREATE EXTENSION fuzzystrmatch; CREATE EXTENSION osml10n; CREATE EXTENSION pg_stat_statements;"
docker-compose run --rm import-osm /usr/src/app/psql.sh -c "GRANT ALL ON SCHEMA public TO public;COMMENT ON SCHEMA public IS 'standard public schema';"
docker-compose run --rm import-osm ./psql.sh -c "DROP SCHEMA IF EXISTS public CASCADE ; CREATE SCHEMA IF NOT EXISTS public; "
docker-compose run --rm import-osm ./psql.sh -c "CREATE EXTENSION hstore; CREATE EXTENSION postgis; CREATE EXTENSION unaccent; CREATE EXTENSION fuzzystrmatch; CREATE EXTENSION osml10n; CREATE EXTENSION pg_stat_statements;"
docker-compose run --rm import-osm ./psql.sh -c "GRANT ALL ON SCHEMA public TO public;COMMENT ON SCHEMA public IS 'standard public schema';"
pgclimb-list-views:
docker-compose run --rm import-osm /usr/src/app/pgclimb.sh -c "select schemaname,viewname from pg_views where schemaname='public' order by viewname;" csv
docker-compose run --rm import-osm ./pgclimb.sh -c "select schemaname,viewname from pg_views where schemaname='public' order by viewname;" csv
pgclimb-list-tables:
docker-compose run --rm import-osm /usr/src/app/pgclimb.sh -c "select schemaname,tablename from pg_tables where schemaname='public' order by tablename;" csv
docker-compose run --rm import-osm ./pgclimb.sh -c "select schemaname,tablename from pg_tables where schemaname='public' order by tablename;" csv
psql-vacuum-analyze:
@echo "Start - postgresql: VACUUM ANALYZE VERBOSE;"
docker-compose run --rm import-osm /usr/src/app/psql.sh -P pager=off -c 'VACUUM ANALYZE VERBOSE;'
docker-compose run --rm import-osm ./psql.sh -P pager=off -c 'VACUUM ANALYZE VERBOSE;'
psql-analyze:
@echo "Start - postgresql: ANALYZE VERBOSE ;"
docker-compose run --rm import-osm /usr/src/app/psql.sh -P pager=off -c 'ANALYZE VERBOSE;'
docker-compose run --rm import-osm ./psql.sh -P pager=off -c 'ANALYZE VERBOSE;'
list-docker-images:
docker images | grep openmaptiles

8
README.md
View File

@ -1,10 +1,10 @@
## OpenMapTiles [![Build Status](https://travis-ci.org/openmaptiles/openmaptiles.svg?branch=master)](https://travis-ci.org/openmaptiles/openmaptiles)
OpenMapTiles is an extensible and open tile schema based on the OpenStreetMap. It is used to generate vector tiles for online zoomable maps. The project is about creating a beautiful basemaps with general layers that contain topographic information. More information [openmaptiles.org](https://openmaptiles.org/) and [openmaptiles.com](https://openmaptiles.com/).
OpenMapTiles is an extensible and open tile schema based on the OpenStreetMap. This project is used to generate vector tiles for online zoomable maps. OpenMapTiles is about creating a beautiful basemaps with general layers containing topographic information. More information [openmaptiles.org](https://openmaptiles.org/) and [openmaptiles.com](https://openmaptiles.com/).
We encourage you to collaborate, reuse and adapt existing layers and add your own layers or use our approach for your own vector tile project. Feel free to fork the repo and experiment. The repository is built on top of the [openmaptiles/openmaptiles-tools](https://github.com/openmaptiles/openmaptiles-tools) to simplify vector tile creation.
We encourage you to collaborate, reuse and adapt existing layers, or add your own layers. You may use our approach for your own vector tile project. Feel free to fork the repo and experiment. The repository is built on top of the [openmaptiles/openmaptiles-tools](https://github.com/openmaptiles/openmaptiles-tools) to simplify vector tile creation.
Please keep in mind that OpenMapTiles schema should stay displaying general topographic content. In case of creating a new layer or expanding an existing one with a specific theme, please create a fork and invite other community members to cooperate on your topic in this separate fork. OpenMapTiles schema is used in many projects all over the world and the size of the final vector tiles needs to be considered in any update.
Please keep in mind that OpenMapTiles schema should display general topographic content. If creating a new layer or expanding an existing layer with a specific theme, please create a fork and invite other community members to cooperate on your topic. OpenMapTiles schema is used in many projects all over the world and the size of the final vector tiles needs to be considered in any update.
- :link: Schema https://openmaptiles.org/schema
- :link: Docs https://openmaptiles.org/docs
@ -76,7 +76,7 @@ To work on OpenMapTiles you need Docker.
Build the tileset.
```bash
git clone git@github.com:openmaptiles/openmaptiles.git
git clone https://github.com/openmaptiles/openmaptiles.git
cd openmaptiles
# Build the imposm mapping, the tm2source project and collect all SQL scripts
make

2
layers/aeroway/mapping.yaml
View File

@ -60,8 +60,6 @@ tables:
type: mapping_value
- name: area
type: area
- name: webmerc_area
type: webmerc_area
mapping:
aeroway: *aeroway_polygon_mapping
"area:aeroway": *aeroway_polygon_mapping

20
layers/boundary/boundary.sql
View File

@ -3,7 +3,10 @@
-- etldoc: ne_110m_admin_0_boundary_lines_land -> boundary_z0
CREATE OR REPLACE VIEW boundary_z0 AS (
SELECT geometry, 2 AS admin_level, false AS disputed, false AS maritime
SELECT geometry,
2 AS admin_level,
(CASE WHEN featurecla LIKE 'Disputed%' THEN true ELSE false END) AS disputed,
false AS maritime
FROM ne_110m_admin_0_boundary_lines_land
);
@ -11,7 +14,10 @@ CREATE OR REPLACE VIEW boundary_z0 AS (
-- etldoc: ne_50m_admin_1_states_provinces_lines -> boundary_z1
CREATE OR REPLACE VIEW boundary_z1 AS (
SELECT geometry, 2 AS admin_level, false AS disputed, false AS maritime
SELECT geometry,
2 AS admin_level,
(CASE WHEN featurecla LIKE 'Disputed%' THEN true ELSE false END) AS disputed,
false AS maritime
FROM ne_50m_admin_0_boundary_lines_land
UNION ALL
SELECT geometry, 4 AS admin_level, false AS disputed, false AS maritime
@ -23,7 +29,10 @@ CREATE OR REPLACE VIEW boundary_z1 AS (
-- etldoc: ne_50m_admin_1_states_provinces_lines -> boundary_z3
CREATE OR REPLACE VIEW boundary_z3 AS (
SELECT geometry, 2 AS admin_level, false AS disputed, false AS maritime
SELECT geometry,
2 AS admin_level,
(CASE WHEN featurecla LIKE 'Disputed%' THEN true ELSE false END) AS disputed,
false AS maritime
FROM ne_50m_admin_0_boundary_lines_land
UNION ALL
SELECT geometry, 4 AS admin_level, false AS disputed, false AS maritime
@ -36,7 +45,10 @@ CREATE OR REPLACE VIEW boundary_z3 AS (
-- etldoc: osm_border_linestring_gen10 -> boundary_z4
CREATE OR REPLACE VIEW boundary_z4 AS (
SELECT geometry, 2 AS admin_level, false AS disputed, false AS maritime
SELECT geometry,
2 AS admin_level,
(CASE WHEN featurecla LIKE 'Disputed%' THEN true ELSE false END) AS disputed,
false AS maritime
FROM ne_10m_admin_0_boundary_lines_land
WHERE featurecla <> 'Lease limit'
UNION ALL

10
layers/building/mapping.yaml
View File

@ -15,8 +15,6 @@ tables:
type: validated_geometry
- name: area
type: area
- name: webmerc_area
type: webmerc_area
- name: material
key: building:material
type: string
@ -78,8 +76,6 @@ tables:
type: validated_geometry
- name: area
type: area
- name: webmerc_area
type: webmerc_area
- name: building
key: building
type: string
@ -172,8 +168,6 @@ tables:
type: validated_geometry
- name: area
type: area
- name: webmerc_area
type: webmerc_area
- name: building
key: building
type: string
@ -266,8 +260,6 @@ tables:
type: validated_geometry
- name: area
type: area
- name: webmerc_area
type: webmerc_area
- name: building
key: building
type: string
@ -360,8 +352,6 @@ tables:
type: validated_geometry
- name: area
type: area
- name: webmerc_area
type: webmerc_area
- name: building
key: building
type: string

55
layers/landcover/landcover.sql
View File

@ -9,82 +9,80 @@
--);
--CREATE INDEX IF NOT EXISTS landcover_grouped_gen2_geometry_idx ON landcover_grouped_gen2 USING gist(geometry);
CREATE OR REPLACE FUNCTION landcover_class(landuse VARCHAR, "natural" VARCHAR, leisure VARCHAR, wetland VARCHAR) RETURNS TEXT AS $$
CREATE OR REPLACE FUNCTION landcover_class(subclass VARCHAR) RETURNS TEXT AS $$
SELECT CASE
WHEN landuse IN ('farmland', 'farm', 'orchard', 'vineyard', 'plant_nursery') THEN 'farmland'
WHEN "natural" IN ('glacier', 'ice_shelf') THEN 'ice'
WHEN "natural"='wood' OR landuse IN ('forest') THEN 'wood'
WHEN "natural" IN ('bare_rock', 'scree') THEN 'rock'
WHEN "natural"='grassland'
OR landuse IN ('grass', 'meadow', 'allotments', 'grassland',
'park', 'village_green', 'recreation_ground')
OR leisure IN ('park', 'garden')
THEN 'grass'
WHEN "natural"='wetland' OR wetland IN ('bog', 'swamp', 'wet_meadow', 'marsh', 'reedbed', 'saltern', 'tidalflat', 'saltmarsh', 'mangrove') THEN 'wetland'
WHEN "natural"IN ('beach', 'sand', 'dune') THEN 'sand'
WHEN subclass IN ('farmland', 'farm', 'orchard', 'vineyard', 'plant_nursery') THEN 'farmland'
WHEN subclass IN ('glacier', 'ice_shelf') THEN 'ice'
WHEN subclass IN ('wood', 'forest') THEN 'wood'
WHEN subclass IN ('bare_rock', 'scree') THEN 'rock'
WHEN subclass IN ('fell', 'grassland', 'heath', 'scrub', 'tundra', 'grass', 'meadow', 'allotments',
'park', 'village_green', 'recreation_ground', 'garden') THEN 'grass'
WHEN subclass IN ('wetland', 'bog', 'swamp', 'wet_meadow', 'marsh', 'reedbed',
'saltern', 'tidalflat', 'saltmarsh', 'mangrove') THEN 'wetland'
WHEN subclass IN ('beach', 'sand', 'dune') THEN 'sand'
ELSE NULL
END;
$$ LANGUAGE SQL IMMUTABLE;
-- etldoc: ne_110m_glaciated_areas -> landcover_z0
CREATE OR REPLACE VIEW landcover_z0 AS (
SELECT NULL::bigint AS osm_id, geometry, NULL::text AS landuse, 'glacier'::text AS "natural", NULL::text AS leisure, NULL::text AS wetland FROM ne_110m_glaciated_areas
SELECT NULL::bigint AS osm_id, geometry, 'glacier'::text AS subclass FROM ne_110m_glaciated_areas
);
CREATE OR REPLACE VIEW landcover_z2 AS (
-- etldoc: ne_50m_glaciated_areas -> landcover_z2
SELECT NULL::bigint AS osm_id, geometry, NULL::text AS landuse, 'glacier'::text AS "natural", NULL::text AS leisure, NULL::text AS wetland FROM ne_50m_glaciated_areas
SELECT NULL::bigint AS osm_id, geometry, 'glacier'::text AS subclass FROM ne_50m_glaciated_areas
UNION ALL
-- etldoc: ne_50m_antarctic_ice_shelves_polys -> landcover_z2
SELECT NULL::bigint AS osm_id, geometry, NULL::text AS landuse, 'ice_shelf'::text AS "natural", NULL::text AS leisure, NULL::text AS wetland FROM ne_50m_antarctic_ice_shelves_polys
SELECT NULL::bigint AS osm_id, geometry, 'ice_shelf'::text AS subclass FROM ne_50m_antarctic_ice_shelves_polys
);
CREATE OR REPLACE VIEW landcover_z5 AS (
-- etldoc: ne_10m_glaciated_areas -> landcover_z5
SELECT NULL::bigint AS osm_id, geometry, NULL::text AS landuse, 'glacier'::text AS "natural", NULL::text AS leisure, NULL::text AS wetland FROM ne_10m_glaciated_areas
SELECT NULL::bigint AS osm_id, geometry, 'glacier'::text AS subclass FROM ne_10m_glaciated_areas
UNION ALL
-- etldoc: ne_10m_antarctic_ice_shelves_polys -> landcover_z5
SELECT NULL::bigint AS osm_id, geometry, NULL::text AS landuse, 'ice_shelf'::text AS "natural", NULL::text AS leisure, NULL::text AS wetland FROM ne_10m_antarctic_ice_shelves_polys
SELECT NULL::bigint AS osm_id, geometry, 'ice_shelf'::text AS subclass FROM ne_10m_antarctic_ice_shelves_polys
);
CREATE OR REPLACE VIEW landcover_z7 AS (
-- etldoc: osm_landcover_polygon_gen7 -> landcover_z7
SELECT osm_id, geometry, landuse, "natural", leisure, wetland FROM osm_landcover_polygon_gen7
SELECT osm_id, geometry, subclass FROM osm_landcover_polygon_gen7
);
CREATE OR REPLACE VIEW landcover_z8 AS (
-- etldoc: osm_landcover_polygon_gen6 -> landcover_z8
SELECT osm_id, geometry, landuse, "natural", leisure, wetland FROM osm_landcover_polygon_gen6
SELECT osm_id, geometry, subclass FROM osm_landcover_polygon_gen6
);
CREATE OR REPLACE VIEW landcover_z9 AS (
-- etldoc: osm_landcover_polygon_gen5 -> landcover_z9
SELECT osm_id, geometry, landuse, "natural", leisure, wetland FROM osm_landcover_polygon_gen5
SELECT osm_id, geometry, subclass FROM osm_landcover_polygon_gen5
);
CREATE OR REPLACE VIEW landcover_z10 AS (
-- etldoc: osm_landcover_polygon_gen4 -> landcover_z10
SELECT osm_id, geometry, landuse, "natural", leisure, wetland FROM osm_landcover_polygon_gen4
SELECT osm_id, geometry, subclass FROM osm_landcover_polygon_gen4
);
CREATE OR REPLACE VIEW landcover_z11 AS (
-- etldoc: osm_landcover_polygon_gen3 -> landcover_z11
SELECT osm_id, geometry, landuse, "natural", leisure, wetland FROM osm_landcover_polygon_gen3
SELECT osm_id, geometry, subclass FROM osm_landcover_polygon_gen3
);
CREATE OR REPLACE VIEW landcover_z12 AS (
-- etldoc: osm_landcover_polygon_gen2 -> landcover_z12
SELECT osm_id, geometry, landuse, "natural", leisure, wetland FROM osm_landcover_polygon_gen2
SELECT osm_id, geometry, subclass FROM osm_landcover_polygon_gen2
);
CREATE OR REPLACE VIEW landcover_z13 AS (
-- etldoc: osm_landcover_polygon_gen1 -> landcover_z13
SELECT osm_id, geometry, landuse, "natural", leisure, wetland FROM osm_landcover_polygon_gen1
SELECT osm_id, geometry, subclass FROM osm_landcover_polygon_gen1
);
CREATE OR REPLACE VIEW landcover_z14 AS (
-- etldoc: osm_landcover_polygon -> landcover_z14
SELECT osm_id, geometry, landuse, "natural", leisure, wetland FROM osm_landcover_polygon
SELECT osm_id, geometry, subclass FROM osm_landcover_polygon
);
-- etldoc: layer_landcover[shape=record fillcolor=lightpink, style="rounded, filled", label="layer_landcover | <z0_1> z0-z1 | <z2_4> z2-z4 | <z5_6> z5-z6 |<z7> z7 |<z8> z8 |<z9> z9 |<z10> z10 |<z11> z11 |<z12> z12|<z13> z13|<z14_> z14+" ] ;
@ -92,11 +90,8 @@ CREATE OR REPLACE VIEW landcover_z14 AS (
CREATE OR REPLACE FUNCTION layer_landcover(bbox geometry, zoom_level int)
RETURNS TABLE(osm_id bigint, geometry geometry, class text, subclass text) AS $$
SELECT osm_id, geometry,
landcover_class(landuse, "natural", leisure, wetland) AS class,
COALESCE(
NULLIF("natural", ''), NULLIF(landuse, ''),
NULLIF(leisure, ''), NULLIF(wetland, '')
) AS subclass
landcover_class(subclass) AS class,
subclass
FROM (
-- etldoc: landcover_z0 -> layer_landcover:z0_1
SELECT * FROM landcover_z0

4
layers/landcover/landcover.yaml
View File

@ -30,13 +30,16 @@ layer:
- beach
- bog
- dune
- scrub
- farm
- farmland
- fell
- forest
- garden
- glacier
- grass
- grassland
- heath
- mangrove
- marsh
- meadow
@ -51,6 +54,7 @@ layer:
- scree
- swamp
- tidalflat
- tundra
- village_green
- vineyard
- wet_meadow

22
layers/landcover/mapping.yaml
View File

@ -51,22 +51,12 @@ tables:
type: id
- name: geometry
type: validated_geometry
- name: landuse
key: landuse
type: string
- name: leisure
key: leisure
type: string
- name: natural
key: natural
type: string
- name: wetland
key: wetland
type: string
- name: area
type: area
- name: webmerc_area
type: webmerc_area
- name: subclass
type: mapping_value
- name: mapping_key
type: mapping_key
mapping:
landuse:
- allotments
@ -86,7 +76,11 @@ tables:
natural:
- wood
- wetland
- fell
- grassland
- heath
- scrub
- tundra
- glacier
- bare_rock
- scree

BIN
layers/landuse/etl_diagram.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 136 KiB

After

Width:  |  Height:  |  Size: 151 KiB

38
layers/landuse/landuse.sql
View File

@ -1,55 +1,61 @@
-- etldoc: ne_50m_urban_areas -> landuse_z4
CREATE OR REPLACE VIEW landuse_z4 AS (
SELECT NULL::bigint AS osm_id, geometry, 'residential'::text AS landuse, NULL::text AS amenity, NULL::text AS leisure, NULL::text AS tourism, NULL::text AS place, scalerank
SELECT NULL::bigint AS osm_id, geometry, 'residential'::text AS landuse, NULL::text AS amenity, NULL::text AS leisure, NULL::text AS tourism, NULL::text AS place, NULL::text AS waterway, scalerank
FROM ne_50m_urban_areas
WHERE scalerank <= 2
);
-- etldoc: ne_50m_urban_areas -> landuse_z5
CREATE OR REPLACE VIEW landuse_z5 AS (
SELECT NULL::bigint AS osm_id, geometry, 'residential'::text AS landuse, NULL::text AS amenity, NULL::text AS leisure, NULL::text AS tourism, NULL::text AS place, scalerank
SELECT NULL::bigint AS osm_id, geometry, 'residential'::text AS landuse, NULL::text AS amenity, NULL::text AS leisure, NULL::text AS tourism, NULL::text AS place, NULL::text AS waterway, scalerank
FROM ne_50m_urban_areas
);
-- etldoc: ne_10m_urban_areas -> landuse_z6
-- etldoc: osm_landuse_polygon_gen7 -> landuse_z6
CREATE OR REPLACE VIEW landuse_z6 AS (
SELECT NULL::bigint AS osm_id, geometry, 'residential'::text AS landuse, NULL::text AS amenity, NULL::text AS leisure, NULL::text AS tourism, NULL::text AS place, scalerank
FROM ne_10m_urban_areas
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, waterway, NULL::int as scalerank
FROM osm_landuse_polygon_gen7
);
-- etldoc: osm_landuse_polygon_gen6 -> landuse_z8
CREATE OR REPLACE VIEW landuse_z8 AS (
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, waterway, NULL::int as scalerank
FROM osm_landuse_polygon_gen6
);
-- etldoc: osm_landuse_polygon_gen5 -> landuse_z9
CREATE OR REPLACE VIEW landuse_z9 AS (
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, NULL::int as scalerank
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, waterway, NULL::int as scalerank
FROM osm_landuse_polygon_gen5
);
-- etldoc: osm_landuse_polygon_gen4 -> landuse_z10
CREATE OR REPLACE VIEW landuse_z10 AS (
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, NULL::int as scalerank
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, waterway, NULL::int as scalerank
FROM osm_landuse_polygon_gen4
);
-- etldoc: osm_landuse_polygon_gen3 -> landuse_z11
CREATE OR REPLACE VIEW landuse_z11 AS (
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, NULL::int as scalerank
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, waterway, NULL::int as scalerank
FROM osm_landuse_polygon_gen3
);
-- etldoc: osm_landuse_polygon_gen2 -> landuse_z12
CREATE OR REPLACE VIEW landuse_z12 AS (
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, NULL::int as scalerank
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, waterway, NULL::int as scalerank
FROM osm_landuse_polygon_gen2
);
-- etldoc: osm_landuse_polygon_gen1 -> landuse_z13
CREATE OR REPLACE VIEW landuse_z13 AS (
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, NULL::int as scalerank
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, waterway, NULL::int as scalerank
FROM osm_landuse_polygon_gen1
);
-- etldoc: osm_landuse_polygon -> landuse_z14
CREATE OR REPLACE VIEW landuse_z14 AS (
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, NULL::int as scalerank
SELECT osm_id, geometry, landuse, amenity, leisure, tourism, place, waterway, NULL::int as scalerank
FROM osm_landuse_polygon
);
@ -64,7 +70,8 @@ RETURNS TABLE(osm_id bigint, geometry geometry, class text) AS $$
NULLIF(amenity, ''),
NULLIF(leisure, ''),
NULLIF(tourism, ''),
NULLIF(place, '')
NULLIF(place, ''),
NULLIF(waterway, '')
) AS class
FROM (
-- etldoc: landuse_z4 -> layer_landuse:z4
@ -77,9 +84,10 @@ RETURNS TABLE(osm_id bigint, geometry geometry, class text) AS $$
UNION ALL
-- etldoc: landuse_z6 -> layer_landuse:z6
-- etldoc: landuse_z6 -> layer_landuse:z7
-- etldoc: landuse_z6 -> layer_landuse:z8
SELECT * FROM landuse_z6
WHERE zoom_level BETWEEN 6 AND 8 AND scalerank-1 <= zoom_level
SELECT * FROM landuse_z6 WHERE zoom_level BETWEEN 6 AND 7
UNION ALL
-- etldoc: landuse_z8 -> layer_landuse:z8
SELECT * FROM landuse_z8 WHERE zoom_level = 8
UNION ALL
-- etldoc: landuse_z9 -> layer_landuse:z9
SELECT * FROM landuse_z9 WHERE zoom_level = 9

4
layers/landuse/landuse.yaml
View File

@ -13,7 +13,8 @@ layer:
[`amenity`](http://wiki.openstreetmap.org/wiki/Key:amenity),
[`leisure`](http://wiki.openstreetmap.org/wiki/Key:leisure),
[`tourism`](http://wiki.openstreetmap.org/wiki/Key:tourism),
or [`place`](http://wiki.openstreetmap.org/wiki/Key:place) tag.
[`place`](http://wiki.openstreetmap.org/wiki/Key:place)
or [`waterway`](http://wiki.openstreetmap.org/wiki/Key:waterway) tag.
values:
- railway
- cemetery
@ -37,6 +38,7 @@ layer:
- zoo
- suburb
- neighbourhood
- dam
datasource:
geometry_field: geometry
query: (SELECT geometry, class FROM layer_landuse(!bbox!, z(!scale_denominator!))) AS t

16
layers/landuse/mapping.yaml
View File

@ -1,4 +1,13 @@
generalized_tables:
# etldoc: imposm3 -> osm_landuse_polygon_gen7
landuse_polygon_gen7:
source: landuse_polygon_gen6
tolerance: ZRES7
# etldoc: imposm3 -> osm_landuse_polygon_gen6
landuse_polygon_gen6:
source: landuse_polygon_gen5
sql_filter: area>power(ZRES6,2) AND (landuse='residential' OR place='suburb' OR place='neighbourhood')
tolerance: ZRES8
# etldoc: imposm3 -> osm_landuse_polygon_gen5
landuse_polygon_gen5:
source: landuse_polygon_gen4
@ -49,10 +58,11 @@ tables:
- name: place
key: place
type: string
- name: waterway
key: waterway
type: string
- name: area
type: area
- name: webmerc_area
type: webmerc_area
mapping:
landuse:
- railway
@ -83,3 +93,5 @@ tables:
place:
- suburb
- neighbourhood
waterway:
- dam

BIN
layers/landuse/mapping_diagram.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 57 KiB

After

Width:  |  Height:  |  Size: 59 KiB

2
layers/park/mapping.yaml
View File

@ -79,8 +79,6 @@ tables:
type: string
- name: area
type: area
- name: webmerc_area
type: webmerc_area
mapping:
leisure:
- nature_reserve

2
layers/place/mapping.yaml
View File

@ -77,8 +77,6 @@ tables:
type: geometry
- name: area
type: area
- name: webmerc_area
type: webmerc_area
- *name
- *name_en
- *name_de

2
layers/transportation/layer.sql
View File

@ -366,7 +366,7 @@ indoor INT, bicycle TEXT, foot TEXT, horse TEXT, mtb_scale TEXT, surface TEXT) A
WHERE zoom_level >= 13
AND (
man_made IN ('bridge', 'pier')
OR (is_area AND COALESCE(layer, 0) >= 0)
OR (ST_GeometryType(geometry) IN ('ST_Polygon','ST_MultiPolygon') AND COALESCE(layer, 0) >= 0)
)
) AS zoom_levels
WHERE geometry && bbox

BIN
layers/transportation_name/mapping_diagram.png
View File

Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 80 KiB

After

Width:  |  Height:  |  Size: 114 KiB

13
layers/water/mapping.yaml
View File

@ -36,6 +36,15 @@ generalized_tables:
sql_filter: area>power(ZRES10,2) AND ST_IsValid(geometry)
tolerance: ZRES12
tunnel_field: &tunnel
key: tunnel
name: is_tunnel
type: bool
bridge_field: &bridge
key: bridge
name: is_bridge
type: bool
tables:
# etldoc: imposm3 -> osm_water_polygon
@ -47,8 +56,6 @@ tables:
type: validated_geometry
- name: area
type: area
- name: webmerc_area
type: webmerc_area
- key: name
name: name
type: string
@ -72,6 +79,8 @@ tables:
- name: is_intermittent
key: intermittent
type: bool
- *tunnel
- *bridge
filters:
reject:
covered: ["yes"]

98
layers/water/water.sql
View File

@ -8,125 +8,160 @@ CREATE OR REPLACE FUNCTION water_class(waterway TEXT) RETURNS TEXT AS $$
$$ LANGUAGE SQL IMMUTABLE;
CREATE OR REPLACE FUNCTION waterway_brunnel(is_bridge BOOL, is_tunnel BOOL) RETURNS TEXT AS $$
SELECT CASE
WHEN is_bridge THEN 'bridge'
WHEN is_tunnel THEN 'tunnel'
ELSE NULL
END;
$$ LANGUAGE SQL IMMUTABLE STRICT;
CREATE OR REPLACE VIEW water_z0 AS (
-- etldoc: ne_110m_ocean -> water_z0
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM ne_110m_ocean
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_110m_ocean
UNION ALL
-- etldoc: ne_110m_lakes -> water_z0
SELECT geometry, 'lake'::text AS class, NULL::boolean AS is_intermittent FROM ne_110m_lakes
SELECT geometry, 'lake'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_110m_lakes
);
CREATE OR REPLACE VIEW water_z1 AS (
-- etldoc: ne_110m_ocean -> water_z1
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM ne_110m_ocean
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_110m_ocean
UNION ALL
-- etldoc: ne_110m_lakes -> water_z1
SELECT geometry, 'lake'::text AS class, NULL::boolean AS is_intermittent FROM ne_110m_lakes
SELECT geometry, 'lake'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_110m_lakes
);
CREATE OR REPLACE VIEW water_z2 AS (
-- etldoc: ne_50m_ocean -> water_z2
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM ne_50m_ocean
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_50m_ocean
UNION ALL
-- etldoc: ne_50m_lakes -> water_z2
SELECT geometry, 'lake'::text AS class, NULL::boolean AS is_intermittent FROM ne_50m_lakes
SELECT geometry, 'lake'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_50m_lakes
);
CREATE OR REPLACE VIEW water_z4 AS (
-- etldoc: ne_50m_ocean -> water_z4
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM ne_50m_ocean
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_50m_ocean
UNION ALL
-- etldoc: ne_50m_lakes -> water_z4
SELECT geometry, 'lake'::text AS class, NULL::boolean AS is_intermittent FROM ne_50m_lakes
SELECT geometry, 'lake'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_50m_lakes
);
CREATE OR REPLACE VIEW water_z5 AS (
-- etldoc: ne_10m_ocean -> water_z5
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM ne_10m_ocean
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_10m_ocean
UNION ALL
-- etldoc: ne_10m_lakes -> water_z5
SELECT geometry, 'lake'::text AS class, NULL::boolean AS is_intermittent FROM ne_10m_lakes
SELECT geometry, 'lake'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_10m_lakes
);
CREATE OR REPLACE VIEW water_z6 AS (
-- etldoc: ne_10m_ocean -> water_z6
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM ne_10m_ocean
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_10m_ocean
UNION ALL
-- etldoc: osm_water_polygon_gen6 -> water_z6
SELECT geometry, water_class(waterway) AS class, is_intermittent FROM osm_water_polygon_gen6
SELECT geometry, water_class(waterway) AS class, is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM osm_water_polygon_gen6
WHERE "natural" != 'bay'
);
CREATE OR REPLACE VIEW water_z7 AS (
-- etldoc: ne_10m_ocean -> water_z7
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM ne_10m_ocean
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM ne_10m_ocean
UNION ALL
-- etldoc: osm_water_polygon_gen5 -> water_z7
SELECT geometry, water_class(waterway) AS class, is_intermittent FROM osm_water_polygon_gen5
SELECT geometry, water_class(waterway) AS class, is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM osm_water_polygon_gen5
WHERE "natural" != 'bay'
);
CREATE OR REPLACE VIEW water_z8 AS (
-- etldoc: osm_ocean_polygon_gen4 -> water_z8
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM osm_ocean_polygon_gen4
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM osm_ocean_polygon_gen4
UNION ALL
-- etldoc: osm_water_polygon_gen4 -> water_z8
SELECT geometry, water_class(waterway) AS class, is_intermittent FROM osm_water_polygon_gen4
SELECT geometry, water_class(waterway) AS class, is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM osm_water_polygon_gen4
WHERE "natural" != 'bay'
);
CREATE OR REPLACE VIEW water_z9 AS (
-- etldoc: osm_ocean_polygon_gen3 -> water_z9
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM osm_ocean_polygon_gen3
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM osm_ocean_polygon_gen3
UNION ALL
-- etldoc: osm_water_polygon_gen3 -> water_z9
SELECT geometry, water_class(waterway) AS class, is_intermittent FROM osm_water_polygon_gen3
SELECT geometry, water_class(waterway) AS class, is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel FROM osm_water_polygon_gen3
WHERE "natural" != 'bay'
);
CREATE OR REPLACE VIEW water_z10 AS (
-- etldoc: osm_ocean_polygon_gen2 -> water_z10
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM osm_ocean_polygon_gen2
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM osm_ocean_polygon_gen2
UNION ALL
-- etldoc: osm_water_polygon_gen2 -> water_z10
SELECT geometry, water_class(waterway) AS class, is_intermittent FROM osm_water_polygon_gen2
SELECT geometry, water_class(waterway) AS class, is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel FROM osm_water_polygon_gen2
WHERE "natural" != 'bay'
);
CREATE OR REPLACE VIEW water_z11 AS (
-- etldoc: osm_ocean_polygon_gen1 -> water_z11
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM osm_ocean_polygon_gen1
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM osm_ocean_polygon_gen1
UNION ALL
-- etldoc: osm_water_polygon_gen1 -> water_z11
SELECT geometry, water_class(waterway) AS class, is_intermittent FROM osm_water_polygon_gen1
SELECT geometry, water_class(waterway) AS class, is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM osm_water_polygon_gen1
WHERE "natural" != 'bay'
);
CREATE OR REPLACE VIEW water_z12 AS (
-- etldoc: osm_ocean_polygon_gen1 -> water_z12
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM osm_ocean_polygon
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM osm_ocean_polygon
UNION ALL
-- etldoc: osm_water_polygon -> water_z12
SELECT geometry, water_class(waterway) AS class, is_intermittent FROM osm_water_polygon
SELECT geometry, water_class(waterway) AS class, is_intermittent, is_bridge, is_tunnel
FROM osm_water_polygon
WHERE "natural" != 'bay'
);
CREATE OR REPLACE VIEW water_z13 AS (
-- etldoc: osm_ocean_polygon -> water_z13
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM osm_ocean_polygon
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM osm_ocean_polygon
UNION ALL
-- etldoc: osm_water_polygon -> water_z13
SELECT geometry, water_class(waterway) AS class, is_intermittent FROM osm_water_polygon
SELECT geometry, water_class(waterway) AS class, is_intermittent, is_bridge, is_tunnel
FROM osm_water_polygon
WHERE "natural" != 'bay'
);
CREATE OR REPLACE VIEW water_z14 AS (
-- etldoc: osm_ocean_polygon -> water_z14
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent FROM osm_ocean_polygon
SELECT geometry, 'ocean'::text AS class, NULL::boolean AS is_intermittent, NULL::boolean AS is_bridge, NULL::boolean AS is_tunnel
FROM osm_ocean_polygon
UNION ALL
-- etldoc: osm_water_polygon -> water_z14
SELECT geometry, water_class(waterway) AS class, is_intermittent FROM osm_water_polygon
SELECT geometry, water_class(waterway) AS class, is_intermittent, is_bridge, is_tunnel
FROM osm_water_polygon
WHERE "natural" != 'bay'
);
@ -134,8 +169,11 @@ CREATE OR REPLACE VIEW water_z14 AS (
-- etldoc: label="layer_water |<z0> z0|<z1>z1|<z2>z2|<z3>z3 |<z4> z4|<z5>z5|<z6>z6|<z7>z7| <z8> z8 |<z9> z9 |<z10> z10 |<z11> z11 |<z12> z12|<z13> z13|<z14_> z14+" ] ;
CREATE OR REPLACE FUNCTION layer_water (bbox geometry, zoom_level int)
RETURNS TABLE(geometry geometry, class text, intermittent int) AS $$
SELECT geometry, class::text, is_intermittent::int AS intermittent FROM (
RETURNS TABLE(geometry geometry, class text, brunnel text, intermittent int) AS $$
SELECT geometry, class::text,
waterway_brunnel(is_bridge, is_tunnel) AS brunnel,
is_intermittent::int AS intermittent
FROM (
-- etldoc: water_z0 -> layer_water:z0
SELECT * FROM water_z0 WHERE zoom_level = 0
UNION ALL

8
layers/water/water.yaml
View File

@ -21,9 +21,15 @@ layer:
description: |
Mark with `1` if it is an [intermittent](http://wiki.openstreetmap.org/wiki/Key:intermittent) water polygon.
values: [0, 1]
brunnel:
description: |
Identifies the type of crossing as either a bridge or a tunnel.
values:
- bridge
- tunnel
buffer_size: 4
datasource:
query: (SELECT geometry, class, intermittent FROM layer_water(!bbox!, z(!scale_denominator!))) AS t
query: (SELECT geometry, class, intermittent, brunnel FROM layer_water(!bbox!, z(!scale_denominator!))) AS t
schema:
- ./water.sql
datasources:

9
layers/waterway/waterway.yaml
View File

@ -1,10 +1,13 @@
layer:
id: "waterway"
description: |
OpenStreetMap [waterways](https://wiki.openstreetmap.org/wiki/Waterways) for higher zoom levels
and Natural Earth rivers and lake centerlines for low zoom levels.
OpenStreetMap [waterways](https://wiki.openstreetmap.org/wiki/Waterways) for higher zoom levels (z9 and more)
and Natural Earth rivers and lake centerlines for low zoom levels (z3 - z8).
Linestrings without a name or which are too short are filtered
out at low zoom levels. Waterways do no not have a `subclass` field.
out at low zoom levels.
Till z11 there is `river` class only, in z12 there is also `canal` generated,
starting z13 there is no generalization according to `class` field applied.
Waterways do not have a `subclass` field.
buffer_size: 4
fields:
name: |

51
qa/table_sizes.py
View File

@ -6,7 +6,6 @@ import subprocess
parser = argparse.ArgumentParser()
parser.add_argument('--noan', action='store_true', help='Not to run make psql-analyze')
TOTAL_SIZE_SQL = """SELECT
pg_size_pretty(sum(size)) AS size
FROM (
@ -18,7 +17,6 @@ FROM (
) a
;""".replace('\"', '\\\"')
TABLE_SIZES_SQL = """SELECT
a.relname as "table",
pg_table_size(a.relid) as "size",
@ -30,7 +28,6 @@ WHERE
ORDER BY a.relname;
""".replace('\"', '\\\"')
TABLES_SQL = """SELECT
a.relname
FROM pg_catalog.pg_statio_user_tables a
@ -59,34 +56,38 @@ COLUMNS_SQL = """select
from {1} t;
""".replace('\"', '\\\"')
def print_column_sizes(tables):
for table in tables:
print "Column sizes of table "+table
print("Column sizes of table " + table)
cmds = [
'docker-compose run --rm import-osm',
'/usr/src/app/psql.sh -t -A -F\",\" -P pager=off',
'-c \"' + COLUMN_NAMES_SQL.format(table).replace('\n', ' ').replace('\r', '') + '\"'
'-c \"' + COLUMN_NAMES_SQL.format(table).replace('\n', ' ').replace('\r',
'') + '\"'
]
# print " ".join(cmds)
output = subprocess.check_output(" ".join(cmds), shell=True)
columns = filter(lambda c: len(c)>0, map(lambda l: l.strip(), output.split('\n')))
columns = filter(lambda c: len(c) > 0,
map(lambda l: l.strip(), output.split('\n')))
# print columns
col_sql = ",\n".join(map(lambda c: "sum(pg_column_size(\\\""+c+"\\\")) as \\\""+c+"\\\"", columns))
col_sql = ",\n".join(
map(lambda c: "sum(pg_column_size(\\\"" + c + "\\\")) as \\\"" + c + "\\\"",
columns))
# print COLUMNS_SQL.format(col_sql, table);
cmds = [
'docker-compose run --rm import-osm',
'/usr/src/app/psql.sh -F\",\" --no-align -P pager=off',
'-c \"' + COLUMNS_SQL.format(col_sql, table).replace('\n', ' ').replace('\r', '') + '\"'
'-c \"' + COLUMNS_SQL.format(col_sql, table).replace('\n', ' ').replace(
'\r', '') + '\"'
]
# print " ".join(cmds)
col_csv = subprocess.check_output(" ".join(cmds), shell=True)
print col_csv
print(col_csv)
if __name__ == "__main__":
@ -94,12 +95,11 @@ if __name__ == "__main__":
try:
if(not args.noan):
print "Running make psql-analyze"
if not args.noan:
print("Running make psql-analyze")
subprocess.check_output("make psql-analyze", shell=True)
print "Total size of tables"
print("Total size of tables")
cmds = [
'docker-compose run --rm import-osm',
'/usr/src/app/psql.sh -F\",\" --no-align -P pager=off',
@ -107,11 +107,10 @@ if __name__ == "__main__":
]
# print " ".join(cmds)
TOTAL_SIZE_CSV = subprocess.check_output(" ".join(cmds), shell=True)
print TOTAL_SIZE_CSV
print "\n"
print(TOTAL_SIZE_CSV)
print("\n")
print "Table sizes"
print("Table sizes")
cmds = [
'docker-compose run --rm import-osm',
'/usr/src/app/psql.sh -F\",\" --no-align -P pager=off',
@ -119,11 +118,10 @@ if __name__ == "__main__":
]
# print " ".join(cmds)
TABLE_SIZES_CSV = subprocess.check_output(" ".join(cmds), shell=True)
print TABLE_SIZES_CSV
print "\n"
print(TABLE_SIZES_CSV)
print("\n")
print "Column sizes"
print("Column sizes")
cmds = [
'docker-compose run --rm import-osm',
'/usr/src/app/psql.sh -t -A -F\",\" -P pager=off',
@ -131,11 +129,12 @@ if __name__ == "__main__":
]
# print " ".join(cmds)
output = subprocess.check_output(" ".join(cmds), shell=True)
tables = filter(lambda t: len(t)>0, map(lambda l: l.strip(), output.split('\n')))
tables = filter(lambda t: len(t) > 0,
map(lambda l: l.strip(), output.split('\n')))
print_column_sizes(tables);
# print tables
except subprocess.CalledProcessError, e:
print "Error:\n", e.output
except subprocess.CalledProcessError as e:
print("Error:\n", e.output)
sys.exit(0)