1096 Commits

Author SHA1 Message Date
Brian Sperlongano
9e4be3e3b0
Remove unused network type params (#1155)
Changes in #1143 removed the need for `name` and `ref` parameters in the network-to-network_type conversion function.  This PR removes this dead code.

Verified by running ./quickstart rhode-island which completed successfully.
2021-07-26 16:37:12 +02:00
Brian Sperlongano
951aa907b2
Fix SQL failures in transportation_name update code (#1154)
This PR fixed bugs introduced in #1147 and #1119 which broke the update triggers in `transportation_name_update`.  I noticed this issue while I was working a different PR and tried to update a table in the `transportation` layer.

**Test**

The following code currently fails because of cascading update failures, but will complete successfully after this PR is merged.
```
UPDATE osm_highway_linestring hl
    SET network = rm.network_type
    FROM osm_route_member rm
    WHERE hl.osm_id=rm.member;
```
2021-07-26 15:22:36 +02:00
Brian Sperlongano
b011b27e52
Implement concurrent highway routes (#1152)
Fixes #1128

This PR adds 6 new columns to the `transportation_name` column, named `route_1` through `route_6`.  These columns contain route information for a section of roadway.  The value is stored in the form **network=ref**.  For example, Interstate 95 in the United States would be `US:I=95`.  Thus, each `route_N` value contains enough information to render a highway shield.  Since a section of road can be a part of more than one route, the `route_2`, `route_3`, etc, will contain the 2nd, 3rd, etc., concurrent routes.

The technical approach was to extend the change in #1135, which added ordered concurrency indexes to the `osm_route_member` table by joining up to the the first six entries to the `osm_transportation_name_network` table.  In addition, that PR provided a ranking system for concurrent highways, ordering first by `network_type` (for example, `us-interstate`, `us-state`, etc), then alphabetically by network name, and then by ref in ascending order.  This ordering of concurrent route memberships is now exposed in this PR in the sequential `route_N` values, meaning that rendered concurrent highway shields will be reasonably sorted.

The renderings below were generated using this branch of OpenMapTiles, as well as a separate branch of openstreetmap-americana:
https://github.com/ZeLonewolf/openstreetmap-americana/tree/openmaptilers-new-features-test

The rendering approach is to use the [formatted expressions](https://maplibre.org/maplibre-gl-js-docs/style-spec/expressions/) feature in mapLibre to insert images into a string of text.  Blank shields are added to the sprite sheet for all possible route networks.  Next, a [styleimagemissing](https://maplibre.org/maplibre-gl-js-docs/api/map/) callback is registered.  As each shield ID is requested, the callback retrieves the sprite shield blank associated with the route's network, draw the `ref` text on the shield, and insert the complete shield back into the map.

Of note, this approach currently results in shields which are rotated about the road rather than being viewport aligned.  This issue is currently documented as maplibre/maplibre-gl-js#188.  A separate repository (https://github.com/ZeLonewolf/maplibre-shield-rotation-sample) has been created as a test case to fix this rotation issue.

Adding route concurrency information to OpenMapTiles would be a major step forward in achieving comprehensive highway shield renderings in a vector map!

**Renderings**:

![routes_1](https://user-images.githubusercontent.com/3254090/126054350-fa7475a7-1b60-4989-bbc2-107678e6c73b.png)
![routes_2](https://user-images.githubusercontent.com/3254090/126054351-fe73bc70-d75f-4ab5-8365-0ee3c3d3eab0.png)
![routes_3](https://user-images.githubusercontent.com/3254090/126054353-a1e74c8f-df21-423c-a300-b7f1a7c9231c.png)
![routes_4](https://user-images.githubusercontent.com/3254090/126054355-6b5dcc83-c611-42b3-bb67-d4f26d789744.png)
2021-07-26 14:28:36 +02:00
Brian Sperlongano
3c15679555
Update brunnel aggregation to avoid splitting highways (#1141)
Fixes #1131

This change does the following:
1. Excludes roads from `transportation_name` that don't have a `name` or a `ref`
2. Updates the road name merging logic to exclude changes in `brunnel` status.  This will ensure that minor bridges don't disrupt the continuity of named roads as the map zooms out.
3. The `brunnel` tag will now only be set when a bridge or tunnel is distinctly named.  Distinctly named is defined as "has a different name from the road on either side".

This example shows an unnamed interstate highway rendered as a continuous feature at low zoom.  This road has many small bridges along its length:

![image](https://user-images.githubusercontent.com/3254090/124370289-b30faa80-dc43-11eb-80d6-034c18ce99ad.png)


This example shows a named bridge rendered with `brunnel` tag set:

![image](https://user-images.githubusercontent.com/3254090/124370298-d0dd0f80-dc43-11eb-8a78-183420a6bd62.png)
2021-07-13 14:44:59 +02:00
Brian Sperlongano
197ea39ae3
Remove unused osm_id column (#1147)
Fixes #1146

This PR removes the always-null osm_id column from the `transportation_name_linestring` table and the series of generalized tables that derive from it.

Demonstration of `transportation_name` objects behaving normally after the column has been removed:

![image](https://user-images.githubusercontent.com/3254090/124684944-4512ef80-de9e-11eb-998c-b66bc23be09e.png)
2021-07-13 13:39:28 +02:00
Frédéric Rodrigo
a851f2c9e9
Review index on osm_waterway_linestring on osm_important_waterway_linestring_gen_z* (#1130)
Review index waterway.

* Remove not used index
* Fit index `osm_waterway_linestring_waterway_partial_idx` to query
2021-07-09 09:35:46 +02:00
Brian Sperlongano
4eb240466e
Simplify Trans-Canada Highway logic (#1143)
Fixes #1142

Screenshot from Alberta showing TCH still shown in `transportation_name` layer:
![image](https://user-images.githubusercontent.com/3254090/124532399-77f5ae80-ddde-11eb-8929-36970ebb8386.png)

Screenshot from Ontario showing TCH still shown in `transportation_name` layer:
![image](https://user-images.githubusercontent.com/3254090/124532512-b68b6900-ddde-11eb-9b6a-636c872375c1.png)
2021-07-06 08:43:26 +02:00
Brian Sperlongano
c8377d38ac
include route relation ref in existence check (#1139)
Fixes #1138
Unblocks #1131
Unblocks #1128 

Below is an example of a way that is part of a route relation (in this case, RI-3), but does not have a ref set on the way itself:
![image](https://user-images.githubusercontent.com/3254090/124366616-1509e900-dc1f-11eb-87ab-aadd4f21287d.png)

This screenshot demonstrates that the `transportation_name` object now contains the ref from the route relation:
![image](https://user-images.githubusercontent.com/3254090/124366563-b5abd900-dc1e-11eb-9fd9-96315ddf7682.png)
2021-07-05 11:21:35 +02:00
Brian Sperlongano
1b0b1fd121
Compute separate concurrency_index on osm_route_member (#1135)
Closes #1134 

Adds a `concurrency_index` column to `osm_route_member` as described in #1134.  For example, listing concurrencies on a [segment of the Washington Bridge](https://www.openstreetmap.org/way/43080535) in Providence, Rhode Island, USA:

```
openmaptiles=# select osm_id, network, ref, network_type, name, concurrency_index from osm_route_member where member=43080535 order by concurrency_index asc;
  osm_id  | network | ref | network_type  |     name      | concurrency_index
----------+---------+-----+---------------+---------------+-------------------
 -1694950 | US:I    | 195 | us-interstate | I 195 (RI/MA) |                 1
 -2308410 | US:US   | 6   | us-highway    | US 6 (RI)     |                 2
 -1347856 | US:US   | 1A  | us-highway    | US 1A (RI)    |                 3
 -2309143 | US:US   | 44  | us-highway    | US 44 (RI)    |                 4
(4 rows)
```
2021-07-02 15:00:25 +02:00
Brian Sperlongano
3f70b878e2
nulling subclass when not present (#1132)
PR #1119 (adding support for `highway=motorway_junction`) missed a few cases where NULLIF functions were needed in order to suppress empty-string `subclass` tags from appearing in vector tiles, resulting in unnecessary  `subclass` entries on `transportation_name` objects where it was not needed, for example:
![image](https://user-images.githubusercontent.com/3254090/123889414-7a11c600-d923-11eb-938c-f278b9bf1ffa.png)

This PR adds the missing NULLIF function so that those empty tags are suppressed:
![image](https://user-images.githubusercontent.com/3254090/123889540-b6452680-d923-11eb-8f0b-5c8c03640059.png)
2021-06-30 07:06:22 +02:00
Frédéric Rodrigo
a0847b85f1
Update osm_poi_point only on change (#1129)
After each diff import the table `osm_poi_point` is fully rewritten due to a full update of the field tags. It is now good to do an a update, event if the content does not change, postgres delete and reinsert internally the record. Resulting in more write and internal table size raising.

Note: not directly linked, but there is a problem in this case, the autocaccum is not sufficient to keep this table size moderate, but grow indefinitely.
2021-06-29 15:05:45 +02:00
Tomas Pohanka
9600cecf00
Add notice into README about import-osm (#1118)
Adding information when run `make import-osm`.

closes https://github.com/openmaptiles/openmaptiles/issues/1116
2021-06-28 22:54:00 +02:00
Frédéric Rodrigo
3b8248af63
Switch from amenity=embassy to office=diplomatic #1084 (#1085)
Switch from amenity=embassy to office=diplomatic as was about here #1084.
2021-06-28 16:58:21 +02:00
Brian Sperlongano
79c2ec929d
Add new layer to serve highway=motorway_junction nodes (#1119)
This PR adds a layer for `highway=motorway_junction` features.

This implementation of highway exits in the transportation_name layer add to the existing layer table structure, and renames the internal column name from "construction" (which was already overloaded with non-construction usages) to "subclass", which will be less confusing to future developers.  The string 'junction' is used as the universal sub-class for highway exits.

A new documentation PR has been opened at openmaptiles/www.openmaptiles.org#69 to reflect these changes in the documentation.
2021-06-28 14:35:27 +02:00
Brian Sperlongano
8d91cb3d94
Improve "make help" output (#1121)
This PR makes three improvements to the current `make help` output:

1. Adds additional help entries for useful `make` targets that were not currently in `make help`
2. Pipes the help output through `less` so that the user can scroll up or down through the various `make` targets if it overflows the length of the user's screen
3. Re-groups the "developer" targets into separate entries for database and docker-related commands

In addition, an unneeded whitespace was removed from line 7 of the Makefile.
2021-06-09 12:36:27 +02:00
Tomas Pohanka
da88949067
WSL folder recommend (#1110)
Add a recommendation for folder use in WSL into README.

Extend #1108
2021-05-26 17:37:33 +02:00
Yuri Astrakhan
ae9498547d
Workflow updates - bump vers, print perfcache info (#1111) 2021-04-30 11:43:04 -04:00
Tomas Pohanka
d0ebdde458
Fix transportation road segments disconnection (#1109)
To avoid discontinuous transportation lines between zooms 9 and 11. 
 - Originally limit geometry by length for z9 - z11 (`ST_Length(geometry) > ZRes(11)`)
 - highway z9 to z11 was generalized during import-osm
 - now just create a filtered and generalized z11 table
 - then merge segments in the same way as from (full-featured) osm_highway_linestring and used this merged z11 for mat.view z10 and z9

Close #1107
2021-04-30 15:39:56 +02:00
zstadler
68ec5c7ac8
Update makefile to catch WSL folder error (#1108)
- Resolve #1103
- Remove dead code
2021-04-30 07:57:12 +02:00
Jeremias Volker
76d5d753d8
Fix own OSM instructions (#1101)
Before it was throwing `ERROR:  make generate-bbox-file area=<area-id>` and `Area source has not been specified. Auto-detecting...
ID 'my' was not found in Geofabrik.` `make: *** [download] Error 1`

* Use more meaningful area name

* rename user data example

Co-authored-by: Tomas Pohanka <TomPohys@gmail.com>
2021-04-20 14:59:43 +02:00
Eva Jelinkova
f4a1a2537a
Water - add ogc_fid for NE sources, edit etldoc (#1098)
1) Adding ogc_fid column into materialized views of NE lakes (for having option to filter or compare)
2) Edit bug in etldoc

* add ogc_fid for NE sources, edit etldoc

* etl_diagram update
2021-04-12 12:09:45 +02:00
Rémy Léone
231a1d192c
update docker-compose version (#1091)
update docker-compose version from 2.3 to 3
2021-04-09 13:42:58 +02:00
Yuri Astrakhan
1f6eb7b4a3
Improve logging - show BBOX if exists (#1096)
The generate-bbox target will now show the value of BBOX if it is already generated.
2021-03-30 21:02:34 +02:00
Rémy Léone
d5480d2369
Update naming convention for networks (#1093)
* Update naming convention for networks

* move to postgres
2021-03-30 08:46:42 +02:00
zstadler
2943e9dc8d
Avoid grep errors in clean-unnecessary-docker (#1089)
When `make clean-unnecessary-docker` is run twice, or when there are no docker images tagges as "<none>", make fails because grep fails:

```
Deleting unnecessary container(s)...
Deleting unnecessary image(s)...
make: *** [Makefile:562: clean-unnecessary-docker] Error 1
```

Using `awk` to _also_ search for "<none>", instead of `grep`, solves this issue.
2021-03-16 17:20:25 +01:00
Tomas Pohanka
fadc4a3ba6
Explicit mention for using iso_a2 attribute. (#1073)
Resolve #939
2021-03-16 11:21:25 +01:00
Tomas Pohanka
bf9e6a19cd
osm_route_member bugfix (#1088)
`!=` is not handling `NULL` values.

Then all networks are `road` and not e.g. `us-interstate` or `ca-transcanada`.
2021-03-16 10:55:26 +01:00
Frédéric Rodrigo
50d569dfee
Remove useless indices on osm_transportation_merge_linestring_genX (#1079)
On creation of materialized views `osm_transportation_merge_linestring_gen_zX`, an index is created each time the new view and not used else where than the next generalization step.
It does not worth it to to a scan on the table to create index that will be used only once, while the next generalisation step can also be done in one full scan.
So, removing these indices.
2021-03-05 09:42:27 +01:00
Adam Laža
d69ae015ad
Mount /export dir in docker-compose.yaml (#1071)
Mount /export dir in docker-compose.yaml
This PR allows `using make generate-tiles-pg` without editing `.env`.
Resolves #1070
Related to PR #1069
2021-02-05 14:58:42 +01:00
Jeroen Hoek
295688eb80
Add place=quarter to city subdivisions (#1065)
This adds `place=quarter` which was introduced in 2011 as a subdivision
of villages/towns/cities as a value that sits between `place=suburb` and
`place=neighbourhood`. It has by now been used over 40,000 times and is
rendered in the OSM-Carto.
2021-01-18 14:47:25 +01:00
Adam Laža
c4c3089f40
Update openmaptiles.yaml - set center to 0,0,1 (#1063) 2021-01-14 18:13:00 +01:00
Adam Laža
4b26c63574
Upgrade version to 3.12.1 (#1062)
* Release 3.12 prerequisite

* Upgrade version.

Co-authored-by: Tomáš <tompohys@gmail.com>
2021-01-14 17:21:43 +01:00
tilmanb
077ec78f63
typo and spelling fixes in the output (#1060)
This simply fixes some spelling and/or grammar errors in the quickstart script output
2021-01-10 16:55:52 +01:00
Adam Laža
461043960e
Bugfix landcover osm (#1061)
* Release 3.12 prerequisite

* Add missing osm_id column to osm_landcover_polygon

Co-authored-by: Tomáš <tompohys@gmail.com>
2021-01-08 15:44:11 +01:00
Sergii Golubev
2aa9630b41
add forgotten 'garages' landuse to the docs (#1059)
In https://github.com/openmaptiles/openmaptiles/pull/720 I've forgotten to add the `garages` class to the docs. This PR is fixing that issue,
2020-12-29 14:26:14 +02:00
Tomas Pohanka
485eb6892c
Release 3.12 prerequisite (#1056) 2020-12-16 14:38:58 +02:00
Adam Laža
5772e61244
Bugfix in osm_landcover_gen_z10. (#1054)
Add missing `NOT` in WHERE condition.
It causes there are duplicated features of subclass `wood` and `forest` and features of other subclasses are missing.
2020-12-14 12:47:45 +02:00
Adam Laža
512b3435ad
Bugfix update_aerodrome_label_point() (#1051)
Bug introduced in #944.
Missing exclamation mark in not equal operator caused that tags were not updated thus there were missing `name_int`, `name:latin` and `name:nonlatin`.
2020-12-08 17:27:12 +02:00
Dalibor Janák
b2b8716840
Update links in readme (#1050)
Updated not existing links to openmaptiles.com.
2020-12-08 16:57:40 +02:00
Taro Matsuzawa aka. btm
924558d7d9
Enable pgquery (#1048)
I try to use pgquery, it needs some configuration in .env file.
I think if we use pgquery instead of mapnik, the performace will be good.
2020-12-08 15:56:01 +02:00
Andrii Korzh
01d53a26c6
Fix detected area filename (#1049)
User can be confused that script detected non existing file.
2020-12-07 17:30:16 +02:00
Adam Laža
77f5d76e37
Refactor schema, one _gen_z view per zoom. (#1045)
* Refactor layer aeroway.

* Refactor layer boundary.

* Refactor layer landcover.

* Refactor layer landuse.

* Refactor layer park.

* Refactor layer transportation.

* Refactor layer water.

* Refactor layer waterway.

* Re-generate water* layers etl_diagrams.

* Regenerate etl_diagrams for waterway.

* Cast NULL to text.
2020-11-26 13:54:00 +01:00
Tomas Pohanka
0c6fe2d9ba
Building aggregation fix (#1044)
This PR solving the speed and memory issue with buildings aggregation on zoom level 13. The problem was to create ST_ClusterDBScan which failed after try to cluster a larger area. Now it will make cluster only in "small" polygon using 'country_osm_grid` which covers the world's lands. 

This PR solving #1022 and #974
2020-11-24 12:27:06 +02:00
Tomas Pohanka
24b9328a5f
Remove unused generalization from waterway (#1043)
During the import-osm step, there are three generalized tables for the waterway that are not used for subsequent processing.
Thanks @lazaa32 for the notice.
2020-11-24 11:30:00 +02:00
Adam Laža
da689f9e42
Fix landcover generalization. (#1042)
This PR brings few modification which make it possible to generalize landcover on larger areas. Tested on Europe.

- Use ST_SnapToGrid to avoid ERROR:  GEOSUnaryUnion: TopologyException: found non-noded intersection between LINESTRING.
- Cluster only landcover subclasses: wood and forest.
- Use less generalized tables as a starting point for next step generalization.
2020-11-24 10:33:06 +02:00
Tomas Pohanka
0776cd3eed
README - Rename target for bbox-file by new tool 5.3 (#1040)
Changed in Use 5.3 - new bbox value (#1012)
2020-11-13 14:34:41 +02:00
Tomas Pohanka
3ef364552e
add state name to national borders (#1039)
This PR adds state name to national borders (`admin_level=2`)

From zoom level 3 there is an added `adm0_l` and `adm0_r` attribute for national borders.

There is 3 main steps: 
1) union border lines from OSM and keep separate LineString between intersects
2) create polygons from step 1), create a point in each state polygon, add to this point a state abbreviation from NaturalEarth Data
3) create a short parallel line on the left and right of a small part of the border in 70m distance, and add information about which state overlap this short line.
2020-11-12 12:14:13 +02:00
Tomas Pohanka
d180988f5f
Move "qa" tools from OMT to OMT-T (#1031)
With the new release of OMT-T (5.3) are available tools [`layer-stat`](https://github.com/openmaptiles/openmaptiles-tools/pull/293). 

With the new release of OMT-T can be replaced `make` target `generate-qareports` by `generate-qa`

Used as:
```
make generate-qa STAT_FUNCTION=frequency LAYER=transportation ATTRIBUTE=class
```
2020-11-12 11:23:06 +02:00
Eva Jelinkova
0aa6648bd9
adding brunnel into z<9 (#1038)
Following https://github.com/openmaptiles/openmaptiles/pull/1027, solving https://github.com/openmaptiles/openmaptiles/issues/999.

Adding brunnel attributes to merge tables of `transportation` layer, now it is possible to style/filter tunnels, bridges and fords in all zoom levels. Tested on Switzerland and Europe. Changes in generation speed and filesize minimal.
2020-11-11 15:37:09 +02:00
Tomas Pohanka
4a1b0afa26
reworked landcover layer (#1035)
Instead of using [Douglas-Peucker algorithm](https://postgis.net/docs/ST_SimplifyPreserveTopology.html) which is using for generalized tables [imposm](https://imposm.org/docs/imposm3/latest/mapping.html#generalized-tables) it is used [Visvalingam-Whyatt algorithm](https://postgis.net/docs/ST_SimplifyVW.html).

Solution:
remove imposm generalized tables (during `import-osm`) and create generalization in `import-sql` step for zooms 7-14. Zooms 0-6 are from Natural Earth data.

Upper zoom levels (7 and 8) are ok to merge with no big impact on creation speed. In Canada from z9 it took too long. Between zoom levels 10 - 13 there can be union polygons with less than 300 edge points (empirical number based on a test on Canada forest, can be discussed). Polygons with more than 300 edge points are just simplified. Zoom 14 is from the original dataset `osm_landcover_polygon`.

There is also removed `osm_id` which is never used and not passed into vector tiles.
2020-11-11 15:11:48 +02:00