17 Commits

Author SHA1 Message Date
zstadler
5e65ac8c52
Add area-base filtering for park names in lower zoom levels (#911)
Resolve https://github.com/openmaptiles/openmaptiles/issues/776

Thanks @zstadler for this PR.
2020-06-09 10:53:18 +02:00
Yuri Astrakhan
6457419e0d
NOOP: Format all layer's SQL code (#917)
I would like to reformat all of our SQL to have a concise coding style.
This makes it far easier to understand the code for a casual contributor,
and lets us spot errors more easily.

Most importantly, it makes it much easier to grep (search) the code because it is more likely to be in the same syntax

Some key changes:
* SQL keywords are always UPPERCASE, e.g. `SELECT WHEN AS END ...`
* types, variables, aliases, and field names (identifiers) are always lower case
* `LANGUAGE 'plpgsql'` is now `LANGUAGE plpgsql` (no quotes)
* a few minor spacing/semicolon cleanups

P.S. Per @TomPohys request, `TABLE` is spelled using upper case despite being a type for consistency with PG Docs. Same for `LANGUAGE SQL` vs `LANGUAGE plpgsql`.
2020-06-08 12:19:55 -04:00
zstadler
2584489ad7
Continue conflict resolution 2020-04-04 13:16:59 +03:00
zstadler
88e9127bfd
Merge branch 'master' into protected_area 2020-04-04 12:50:03 +03:00
Ludovic Delauné
924ccd6015
remove useless sql selection (#752) 2020-03-13 15:14:30 -04:00
Jorge Sanz
ace759590e
Parallel capability to layer functions (#728)
This PR allows queries to be parallelized on recent versions of Postgres. The `PARALLEL SAFE` modifier has been added to the layer functions and a PLPGSQL function to convert strings into number has been replaced.

`PARALLEL SAFE` is a modifier for `CREATE FUNCTION` available since Postgres 9.6, so this change does not break current OpenMapTiles supported database version. More details about this topic [here](https://www.postgresql.org/docs/current/parallel-safety.html) and at the reference documentation for [`CREATE FUNCTION`](https://www.postgresql.org/docs/current/sql-createfunction.html).

### Testing procedure

The procedure to test this was:

* Imported `spain.pbf` in a clean environment
* Dumped the OpenMapTiles database from the Postgres Docker image
* Created a clean Postgres 12 database using the default Docker image
* Installed `postgis` 3 from the default Debian package and `osml10n` 2.5.8 from the repository (`make`, etc.)
* Restored the dump
* Lowered the postgres planner parameters for triggering parallel plans:
```sql
set parallel_setup_cost = 5;
set parallel_tuple_cost = 0.005;
```
* Manually added the `PARALLEL SAFE` modifier to each function involved in layer queries (not on updates or inserting functions).
* For each layer, run a testing query to confirm parallel workers were created, something like this:
```sql
explain analyze 
select * from layer_aerodrome_label(tilebbox(8,128,95),10,null)
union all
select * from layer_aerodrome_label(tilebbox(8,128,97),10,null);
```
* After all the layers were processed and confirmed to start parallel executions, a more complete example was run. This example just retrieves the geometries for all the layers from the same tile but without using any MVT related function.

<details><summary>Testing query</summary>

```sql
-- Using the function layer_landuse
explain analyze 
select geometry from layer_water(tilebbox(14,8020,6178),14)
union all
select geometry from layer_waterway(tilebbox(14,8020,6178),14)
union all
select geometry from layer_landcover(tilebbox(14,8020,6178),14)
union all
select geometry from layer_landuse(tilebbox(14,8020,6178),14)
union all
select geometry from layer_mountain_peak(tilebbox(14,8020,6178),14)
union all
select geometry from layer_park(tilebbox(14,8020,6178),14)
union all
select geometry from layer_boundary(tilebbox(14,8020,6178),14)
union all
select geometry from layer_aeroway(tilebbox(14,8020,6178),14)
union all
select geometry from layer_transportation(tilebbox(14,8020,6178),14)
union all
select geometry from layer_building(tilebbox(14,8020,6178),14)
union all
select geometry from layer_water_name(tilebbox(14,8020,6178),14)
union all
select geometry from layer_transportation_name(tilebbox(14,8020,6178),14)
union all
select geometry from layer_place(tilebbox(14,8020,6178),14)
union all
select geometry from layer_housenumber(tilebbox(14,8020,6178),14)
union all
select geometry from layer_poi(tilebbox(14,8020,6178),14)
union all
select geometry from layer_aerodrome_label(tilebbox(14,8020,6178),14);
```
</details>

You can inspect the execution plan and results on [this page](https://explain.dalibo.com/plan/3z). Also [attaching](https://github.com/openmaptiles/openmaptiles/files/3951822/explain-tile-simple.tar.gz) the query and JSON output for future reference. The website gives a ton of details, but you may want to search for nodes mentioning `workers` or `parallel` like in this area referring to `osm_border` or `osm_aeroway_linestring` entities

![image](https://user-images.githubusercontent.com/188264/70647153-9cac9300-1c48-11ea-96ea-ac7a1e2f4a79.png)

### Next steps

Since the execution plan is not showing a parallel append at the top level, meaning it's not running each layer individually, I want to continue experimenting with parameters and queries to see if it's possible to even parallelize more the request.

I will post my finding here, even no change in the code should happen.


cc. @nyurik

Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
2020-01-31 19:36:02 -05:00
zstadler
5657ede450 Add boundary=protected_area parks
Update the **class** of the **park* layer in a backward-compatible way.

The class for `boundary=protected_area` parks is the lower-case of the
[`protection_title`](http://wiki.openstreetmap.org/wiki/key:protection_title)
value with blanks replaced by `_`.
`national_park` is the class of `protection_title=National Park` and `boundary=national_park`.
`nature_reserve` is the class of `protection_title=Nature Reserve` and `leisure=nature_reserve`.
The class for other [`protection_title`](http://wiki.openstreetmap.org/wiki/key:protection_title)
values is similarly assigned.

Resolve https://github.com/openmaptiles/openmaptiles/issues/760
2020-01-27 11:12:57 +02:00
jirik
fe057eede1 Move geometry && bbox check in park layer 2018-11-30 15:32:26 +01:00
jirik
7cd29e2622 Add extra geometry_point && bbox check to park layer 2018-11-30 15:32:26 +01:00
jirik
cd920755bf Remove park_label layer, add labels to park layer 2018-11-07 18:22:26 +01:00
jirik
ccf8ee6dfa Add park labels layer 2018-11-05 16:40:32 +01:00
Lukas Martinelli
5ec4f0766b Add zoom level range slots to landcover diagram 2016-12-01 08:29:34 +00:00
Lukas Martinelli
c1b1913c7c Fix wrong etl docs 2016-11-30 18:38:28 +00:00
Lukas Martinelli
bcc38a0911 Show large parks already on z6 2016-11-30 17:11:47 +00:00
Lukas Martinelli
0c3b3b8b63 Show large parks on z7 2016-11-30 14:16:09 +00:00
Lukas Martinelli
5ebb33575e Move public parks indicating grass to landcover 2016-11-29 19:38:02 +00:00
Lukas Martinelli
09dddebac4 Add park layer 2016-11-28 19:16:27 +00:00