OpenSeaMap-dev Diskussion:Downloadable Raster Charts: Unterschied zwischen den Versionen

Aus OpenSeaMap-dev
Wechseln zu:Navigation, Suche
(tile server: landez statt perl)
(tile server: py script added)
Zeile 42: Zeile 42:
 
** [https://github.com/makinacorpus/landez landez]  
 
** [https://github.com/makinacorpus/landez landez]  
 
* [http://www.dacust.com/inlandwaters/imgkap/ imgkap]
 
* [http://www.dacust.com/inlandwaters/imgkap/ imgkap]
 +
 +
==== py script ====
 +
* relies on [https://github.com/makinacorpus/landez landez]
 +
 +
<pre>
 +
import logging
 +
import shutil
 +
import math
 +
from landez import MBTilesBuilder
 +
from landez import ImageExporter
 +
 +
logging.basicConfig(level=logging.DEBUG)
 +
 +
# to be imported from sqlite-queue
 +
sqBottomGeoPre = 53.388333333333
 +
sqTopGeoPre = 53.73
 +
sqLeftGeoPre = 7.9483333333333
 +
sqRightGeoPre = 8.35
 +
sqNumTiles16 = 7656
 +
sqChart = "1460"
 +
sqname = "Die Jade, Innerer Teil"
 +
sqUrl = "http://tiles.grade.de/tiles.py/openriverboatmap/{z}/{x}/{y}.png"
 +
sqMicrotime= 1234
 +
 +
# to be calculated from numtiles16, numtiles < 700
 +
# numtiles16 is right now hardcoded at int-chart.gejson for ol depth of view reasons, there might be smarter ways in future
 +
# https://github.com/makinacorpus/landez/issues/44
 +
myNumTiles = sqNumTiles16 * 4 * 4
 +
myZoom = 18
 +
 +
while (myNumTiles > 10):
 +
myNumTiles = myNumTiles / 4
 +
myZoom = myZoom - 1
 +
 +
# settings
 +
myFileName = sqChart
 +
myMbtilesFile = myFileName + ".mbtiles"
 +
myImageFile = myFileName + ".jpg"
 +
myKapFile = myFileName + ".kap"
 +
myTextFile = myFileName + ".txt"
 +
myBbox = (sqLeftGeoPre, sqBottomGeoPre, sqRightGeoPre, sqTopGeoPre)
 +
myZoomLevels = [myZoom]
 +
myZoomLevel = myZoom
 +
myTileSize = 256
 +
 +
# download and mbtiles build
 +
mb = MBTilesBuilder(tiles_url=sqUrl, cache=False, filepath=myMbtilesFile, tile_size=myTileSize)
 +
mb.add_coverage(myBbox, myZoomLevels)
 +
mb.run(force=True)
 +
 +
# export image from mbtiles
 +
# not the best idea: causes IO at temp dir, again
 +
ie = ImageExporter(mbtiles_file=myMbtilesFile)
 +
ie.export_image(myBbox, myZoomLevel, imagepath=myImageFile)
 +
# save the geoinfo
 +
myGrid = ie.grid_tiles(myBbox, myZoomLevel)
 +
sqWidth = len(myGrid[0])
 +
sqHeight = len(myGrid)
 +
sqWidthPix = sqWidth * myTileSize
 +
sqHeightPix = sqHeight * myTileSize
 +
 +
# delete temp dir (ImageExporter does not!)
 +
# bad: path is hard coded. how do I get hold of landez's temp path?
 +
# https://github.com/makinacorpus/landez/issues/42
 +
try:
 +
    shutil.rmtree('/tmp/landez')
 +
except OSError:
 +
    pass
 +
 +
# geoinfo output
 +
logging.debug(" grid: " +  str(myGrid))
 +
 +
myLeftColumn = myGrid[0]
 +
myTopLeft = myLeftColumn[0]
 +
sqLeftTile = myTopLeft[0]
 +
sqTopTile = myTopLeft[-1]
 +
 +
myRightColumn = myGrid[-1]
 +
myBottomRight = myRightColumn[-1]
 +
sqRightTile = myBottomRight[0]
 +
sqBottomTile = myBottomRight[-1]
 +
 +
# calculate coords from tilenumbers
 +
def num2deg(xtile, ytile, zoom):
 +
  n = 2.0 ** zoom
 +
  lon_deg = -180.0 + 360.0 * xtile / n
 +
  lat_rad = math.atan(math.sinh(math.pi * (1.0 - 2.0 * (ytile / n))))
 +
  lat_deg = 180.0 * lat_rad / math.pi
 +
  return (lon_deg, lat_deg)
 +
 +
sqBottomGeoPost = (num2deg(sqRightTile + 1, sqBottomTile + 1, myZoom)[-1])
 +
sqTopGeoPost = (num2deg(sqLeftTile, sqTopTile, myZoom)[-1])
 +
sqLeftGeoPost = (num2deg(sqLeftTile, sqTopTile, myZoom)[0])
 +
sqRightGeoPost = (num2deg(sqRightTile + 1, sqBottomTile + 1, myZoom)[0])
 +
 +
# shall be exported to sqlite-queue
 +
logging.debug(" width: " + str(sqWidth) + " tiles")
 +
logging.debug(" height: " + str(sqHeight) + " tiles")
 +
logging.debug(" widthpix: " + str(sqWidthPix) + " pixel")
 +
logging.debug(" heightpix: "+ str(sqHeightPix) + " pixel")
 +
 +
logging.debug(" left tile " + str(sqLeftTile))
 +
logging.debug(" right tile: " + str(sqRightTile))
 +
logging.debug(" top tile: " + str(sqTopTile))
 +
logging.debug(" bottom tile: "+ str(sqBottomTile))
 +
 +
logging.debug(" left: " +  str(sqLeftGeoPost))
 +
logging.debug(" right: " +  str(sqRightGeoPost))
 +
logging.debug(" top: " +  str(sqTopGeoPost))
 +
logging.debug(" bottom: " +  str(sqBottomGeoPost))
 +
 +
# start imgkap to generate .kap file
 +
import subprocess
 +
print "starting imgkap"
 +
myImgkapCall = "./imgkap " + myImageFile + " " + str(sqTopGeoPost) + " " + str(sqLeftGeoPost) + " " + str(sqBottomGeoPost) + " " + str(sqRightGeoPost) + " " + myKapFile
 +
print myImgkapCall
 +
subprocess.call(myImgkapCall, shell=True)
 +
print "end  imgkap"
 +
</pre>
  
 
== Reason for ChartBundler ==
 
== Reason for ChartBundler ==

Version vom 11. Februar 2015, 12:55 Uhr

ideas

web frontend

  • die INT-chart Grenzen (als Basis fuer ein download-tool) bekomme ich inzwischen auf den Schirm: http://kap.grade.de/int-chart-ol2.html , sicherlich verbesserungsbeduerftig
  • workflow
    • INT-Chart durch klick auf die Karte waehlen
    • download iniziieren
      • Weiterleitung auf neue Seite
      • pruefen, ob und in welchem Format vorhanden
      • wenn ja und nicht zu alt, downloadlinks anbieten
      • wenn nein Job in queue-db schreiben
        • number, name, bbox, scale, sessionid, ip-address, email, microtime, status=open, ...
  • das ganze garnieren mit
    • Pruefung, ob das script fuer diese karte schon in der Warteschlange ist
    • Limitierung auf x downloads/jobs pro session ...
    • email notifier
    • captcha

prerequisites

  • apache
  • php
  • sqlite

tile server

  • cronjob checkt queue-db alle x Minuten
  • waehlt aeltesten job aus queue-db mit status=open
  • tiledownload ueber proxy mit landez
    • sinnvolles zoom-level anhand bbox errechnen (max. 10MB/chart)
    • mbtiles erzeugen (siehe auch mbtiles implementation)
    • stitchen und .png/.jpg erzeugen
    • .kap mit imgkap erstellen
    • ...
  • status=done, zoom-level, date, ... in queue-db setzen
  • temp-dir bereinigen und results in charts-dir kopieren

prerequisites

  • apache
    • mod_python
  • tilestache
  • nginx remote proxy
  • imagemagick
  • python
  • imgkap

py script

import logging
import shutil
import math
from landez import MBTilesBuilder
from landez import ImageExporter

logging.basicConfig(level=logging.DEBUG)

# to be imported from sqlite-queue
sqBottomGeoPre = 53.388333333333
sqTopGeoPre = 53.73
sqLeftGeoPre = 7.9483333333333
sqRightGeoPre = 8.35
sqNumTiles16 = 7656
sqChart = "1460"
sqname = "Die Jade, Innerer Teil"
sqUrl = "http://tiles.grade.de/tiles.py/openriverboatmap/{z}/{x}/{y}.png"
sqMicrotime= 1234

# to be calculated from numtiles16, numtiles < 700
# numtiles16 is right now hardcoded at int-chart.gejson for ol depth of view reasons, there might be smarter ways in future
# https://github.com/makinacorpus/landez/issues/44
myNumTiles = sqNumTiles16 * 4 * 4
myZoom = 18

while (myNumTiles > 10):
 myNumTiles = myNumTiles / 4
 myZoom = myZoom - 1

# settings
myFileName = sqChart
myMbtilesFile = myFileName + ".mbtiles"
myImageFile = myFileName + ".jpg"
myKapFile = myFileName + ".kap"
myTextFile = myFileName + ".txt"
myBbox = (sqLeftGeoPre, sqBottomGeoPre, sqRightGeoPre, sqTopGeoPre)
myZoomLevels = [myZoom]
myZoomLevel = myZoom
myTileSize = 256

# download and mbtiles build
mb = MBTilesBuilder(tiles_url=sqUrl, cache=False, filepath=myMbtilesFile, tile_size=myTileSize)
mb.add_coverage(myBbox, myZoomLevels)
mb.run(force=True)

# export image from mbtiles
# not the best idea: causes IO at temp dir, again
ie = ImageExporter(mbtiles_file=myMbtilesFile)
ie.export_image(myBbox, myZoomLevel, imagepath=myImageFile)
# save the geoinfo
myGrid = ie.grid_tiles(myBbox, myZoomLevel)
sqWidth = len(myGrid[0])
sqHeight = len(myGrid)
sqWidthPix = sqWidth * myTileSize
sqHeightPix = sqHeight * myTileSize

# delete temp dir (ImageExporter does not!)
# bad: path is hard coded. how do I get hold of landez's temp path?
# https://github.com/makinacorpus/landez/issues/42
try:
    shutil.rmtree('/tmp/landez')
except OSError:
    pass

# geoinfo output
logging.debug(" grid: " +  str(myGrid))

myLeftColumn = myGrid[0]
myTopLeft = myLeftColumn[0]
sqLeftTile = myTopLeft[0]
sqTopTile = myTopLeft[-1]

myRightColumn = myGrid[-1]
myBottomRight = myRightColumn[-1]
sqRightTile = myBottomRight[0]
sqBottomTile = myBottomRight[-1]

# calculate coords from tilenumbers
def num2deg(xtile, ytile, zoom):
  n = 2.0 ** zoom
  lon_deg = -180.0 + 360.0 * xtile / n
  lat_rad = math.atan(math.sinh(math.pi * (1.0 - 2.0 * (ytile / n))))
  lat_deg = 180.0 * lat_rad / math.pi
  return (lon_deg, lat_deg)

sqBottomGeoPost = (num2deg(sqRightTile + 1, sqBottomTile + 1, myZoom)[-1])
sqTopGeoPost = (num2deg(sqLeftTile, sqTopTile, myZoom)[-1])
sqLeftGeoPost = (num2deg(sqLeftTile, sqTopTile, myZoom)[0])
sqRightGeoPost = (num2deg(sqRightTile + 1, sqBottomTile + 1, myZoom)[0])

# shall be exported to sqlite-queue
logging.debug(" width: " + str(sqWidth) + " tiles")
logging.debug(" height: " + str(sqHeight) + " tiles")
logging.debug(" widthpix: " + str(sqWidthPix) + " pixel")
logging.debug(" heightpix: "+ str(sqHeightPix) + " pixel")

logging.debug(" left tile " + str(sqLeftTile))
logging.debug(" right tile: " + str(sqRightTile))
logging.debug(" top tile: " + str(sqTopTile))
logging.debug(" bottom tile: "+ str(sqBottomTile))

logging.debug(" left: " +  str(sqLeftGeoPost))
logging.debug(" right: " +  str(sqRightGeoPost))
logging.debug(" top: " +  str(sqTopGeoPost))
logging.debug(" bottom: " +  str(sqBottomGeoPost))

# start imgkap to generate .kap file
import subprocess
print "starting imgkap"
myImgkapCall = "./imgkap " + myImageFile + " " + str(sqTopGeoPost) + " " + str(sqLeftGeoPost) + " " + str(sqBottomGeoPost) + " " + str(sqRightGeoPost) + " " + myKapFile
print myImgkapCall
subprocess.call(myImgkapCall, shell=True)
print "end  imgkap"

Reason for ChartBundler

  • Lets talk a bit about amounts here and not low stress or high stress. Using the charts offline on a mobile device. Lets say the display is about 1024x1024 Pixel, then we will need at least 16 tiles per map. If we start on zoom level 8 with the bounding box and continue to zoom level 16 for the harbours, we need around 2 Mio tiles. This 16 tiles on zoom level 8 is about the northern Adria. If we want the Channel just skipping Lands End up to the belgian coast, this would be 6 by 5 on zoom level 8. --Alexej
  • Hi Alexej, do not understand this paragraph!? 2 test datasets of mine:
    • INT-1463 bei zoom-level 13 (s="53.7" n="54.25" w="7.4" e="9.05"):
      x: 37
      y: 21
      total: 777
      [1]
    • river Ruhr
      zoomlevel=16
      ~ 370 tiles
      [2]
--Kannix (Diskussion) 16:22, 29. Jan. 2015 (UTC)