GeoDienstenCentrum Blog

homepage

2026-05-28T13:23:10+02:00

Voor advies over en implementatie van toegankelijke ruimtelijke informatie.

sitemap

2026-05-28T13:23:10+02:00

Sitemap voor GeoDienstenCentrum Blog

DokuWikiSpatial: spatial indexing and search for your wiki

2003-12-13T09:17:51-08:00

Indexing

Now that we can add location to our wiki pages using the mapping and geotagging plugins discussed in previous posts lets start making use of that data. Next to the SEO benefits it would be nice to be able to use that data for generating information. To start off this data needs to be made searchable in a fast manner, this is done by creating a (spatial) index.

The chosen index algorithm is based on a value calculated from the coordinate pair, the geohash. This hash provides a one dimensional representation of a coordinate pair with the length of the hash being a measure for the accuracy of the coordinates. For example the coordinates (57.64911 10.40744) gives a hash of u4pruydqqvj and (57.6 10.4) gives a hash of u4pru. A big advantage is that this can be easily stored in an array with the geohash as a key and the resource(s) as value. Using PHP’s built-in serialisation it is stored on disk in plain text, just like the other indexes and wiki metadata.

As part of the indexing both a KML and a GeoRSS file are generated, these can be served up as a spatial sitemap but may also be used in a map on the wiki. Also, as DokuWiki provides support for EXIF data in images, when uploading JPEG or TIFF media into the wiki’s media store these are added to the spatial index as well if they have the proper EXIF GPS tags.

Searching

The most typical question for generating information is “What is near (to)?…” In case of the wiki resources this would yield one or more pages or images.

Looking back at the example above note that there is an overlap in the hash, this is another benefit of using the geohash. This allows selective matching implicitly enabling the use of a bounding box search ie. longer geohashes specify a more accurate location but in reverse also a smaller serach area.

A small drawback with using a geohash this way is that the positional error and boundingbox grow rapidly with shortening geohash length. For example, a geohash of five characters (u4pru) has an error of ±2.4km where a geohash that is three characters shorter (u4) has a positional error of ±630km. This means that when doing hash based lookups the actual search location is not static with shortening the hash/enlarging the bounds. This requires some more work, but for the initial 0.1 release of the plugin it is sufficient.

The plugin integrates with the geotag plugin to link to a page with dynamic search results. So a geotag with coordinates 48º11'36.384"N;16º27'39.06"E links to ?do=findnearby&lat=48.19344&lon=16.46085 which initially does a lookup in the index for a geohash of u2ednt67js. This will render as a html page with a list of results. The search interface will be augmented to support other output formats, eg. a complete map or a GeoRSS document are also possibilities.

An example of a list of search results on a wiki page of a findnearby to 50.62ºN;6.04ºE.

Some more examples are available on the sample site and wild-water.nl

Talk about this on twitter.

DokuWikiSpatial: geotagging for your wiki

2003-12-13T09:17:51-08:00

Geotagging

After adding mapping capabilities to your wiki using the openlayersmap plugin we will now look into actually creating spatial data as part of your content using geotagging. Geotagging is the process of adding geographic (meta) data to content, in our case wiki pages.

To accomplish this a syntax plugin was developed which allows adding the relevant data to the page. An example of this syntax is shown in the following code snippet.

{{geotag>lat:48.19344, lon:16.46085, placename:VERBUND-Wasserarena,
    country:AT, region:AT-9}}

This syntax is parsed and rendered as part of the page metadata:

...
<meta name="geo.region" content="AT-9"/>
<meta name="geo.placename" content="VERBUND-Wasserarena"/>
<meta name="geo.position" content="48.19344;16.46085"/>
<meta name="geo.country" content="AT"/>
<meta name="ICBM" content="48.19344, 16.46085"/>
<meta name="geo.geohash" content="u2ednt67js"/>
<meta name="DC.title" content="nieuwe Wildwaterbaan Donau Insel, Wenen"/>
...

This metadata is also stored in the wiki’s metadata system so that it can be indexed and searched. When using a plugin such as socialcards the appropriate OpenGraph elements are also exported in the page headers.

The geotag is rendered as part of the page content/body in the form of a geo microformat and a schema.org Place. This allows indexing of the page location by search engines such as Yandex, Bing and Google.

...
<span class="geotagPrint">Geotag (locatie) voor: </span>
<div class="geo" title="Geotag (locatie) voor VERBUND-Wasserarena"
    itemscope itemtype="http://schema.org/Place">
  <span itemprop="name">VERBUND-Wasserarena</span>:&nbsp;<a
     href="wildwaterbaan_donau_insel_wenen?do=findnearby&amp;lat=48.19344&amp;lon=16.46085"
     title="Zoek in de buurt van VERBUND-Wasserarena">
  <span itemprop="geo" itemscope itemtype="http://schema.org/GeoCoordinates">
  <span class="latitude" itemprop="latitude" content="48.19344">48º11'36.384"N</span>;
  <span class="longitude" itemprop="longitude" content="16.46085">16º27'39.06"E</span>
  <span class="a11y">Zoek in de buurt van VERBUND-Wasserarena</span></a></span>
</div>
...

Which looks something like the figure below in the default DokuWiki layout.

An example of a rendered geotag on a wiki page locating the VERBUND-Wasserarena at 48º11'36.384"N;16º27'39.06"E.

Using the geotags on your pages opens up other ways of exploring and viewing your pages. One example is using the content in an augmented reality application such as Mixare. Another is finding pages that are located nearby to the current page or using a map as a browsing interface for exploration of yout wiki.

DokuWikiSpatial: maps for your wiki

2003-12-13T09:17:51-08:00

Spatially enable your Wiki, part 1: mapping

In this series we will look at spatially enabling the DokuWiki system by adding mapping, data management and spatial indexing with the help of geotagging and other means. In general this functionality is described using the term GeoCMS and are provided as plugins by the DokuwikiSpatial project. In this first post we’ll look at mapping after quick introduction to the base DokuWiki stack.

About DokuWiki

The Dokuwiki software is a plain text driven PHP powered wiki engine with a well defined API and plugin mechanism. A big advantage of having a plain text storage is that you can easily run it from a USB stick (eg. DokuWiki on a stick or other portable methods, another feature is that its easy to set up and manage because there is just the webserver to handle, contrary to most other wiki software that requires some sort of SQL-enabled back-end.

At the time I started using DokuWiki some seven years ago there were already some plugin’s that allowed putting a map in a page, however these did not fit my bill of having free basemap data and user editable content for overlays or points of interest. Also they suffer from major accessibility issues. Having only a little experience using PHP at the time I looked into the plugin system and found a well documented API and extensive list of samples kick-starting me to write my own mapping plugin for the wiki, openlayersmap.

Maps in your wiki

As the name suggests the openlayersmap pluging uses the OpenLayers library. It has a choice of both free and proprietary basemap options OSM or MapQuest and Bing or Google if you really want that.

The latest version (4.0) of the openlayersmap plugin enables users to add wiki markup to a page that contains a simple configuration for the map such as dimensions, initial location and initial scale. It also allows for entering a list of points of interest with a descriptive text and a symbol layout (image, rotation, opacity). Next to having a list of points a GPX, KML or GeoJSON file may be loaded from the wiki’s media directory. The POI data, just like the media files is actually part of the page/wiki so that it can be indexed and queried and the initial location of the map is added to the page metadata so it can be used by internal and external search engines.

An example of the slippy map showing a popup with a photograph of the Gumpen waterfall in the Loisach. The wiki markup for this is listed below

<olmap id="olmap" width="660px" height="400px" lat="47.48" lon="10.98"
   zoom="13" statusbar="1" baselyr="landscape">
47.446383,10.918047,0,.8,stuw.png,Gumpen waterval \\
 {{http://www.hooidonksekanoclub.nl/fotoalbum/2009-05-hemelvaart/slides/IMG_7114.jpg?150|Gumpen waterval gezien van stroomaf"}}
47.452819,10.923728,0,.8,uitstap.png,uitstap bovenloop Loisach
47.482083,10.989167,0,.8,instap.png,Instap Gschwandsteg
47.48146,11.04633,0,.8,uitstap.png, Uitstap "Zielhaus Grainau"
</olmap>

In the rendered page the map is available both as a static image with an associated table listing the points of interest and as slippy map that provides dynamic navigation and information display (progressive enhancement). The slippy map can be used (navigated and queried) using both the keyboard and pointing devices. (For more on this see: Enhancing OpenLayers control accessibility) The static image and table are used when printing and may also be used by screen readers (actually this needs a little more work) and provide a fall-back mechanism should the browser not (properly) support javascript or CSS.

Stay tuned for other posts in this series on:

Talk about this on twitter.

Enhancing OpenLayers control accessibility

2003-12-13T09:17:51-08:00

Most OpenLayers control widgets suffer from poorly designed or missing keyboard interoperability. Some attempts have been made to enhance (1, 2) these shortcomings. However these changes and enhancements have not landed, are difficult to implement or carry a burden of extra javascript frameworks. Also they don’t address some of the semantic issues that are present in OpenLayers generated markup.

I have extended some of the most used controls to be keyboard accessible and have enhanced usability by adding tooltips as part of the markup.

Zoom control

The problem with the original zoom control is that it uses a hash hyperlink (<a href="#" /> element) for something that is an action. Hyperlinks are supposed to navigate somewhere within a page with a consistent manner, eg. clicking the zoom in link should always take you to the same place within the page. This is not the case, clicking the link more than once does not always take you to the same place. The anchor element is really misused to emulate a button, people smarter than me have explained that you can’t create a button and links are not buttons.

Note that the before/after code snippets below have been abbreviated by shortening style, class and id attributes.

Before

Markup of the original:

<div id="..." class="..."
    unselectable="on" style="...">
  <a href="#zoomIn" class="olControlZoomIn olButton">+</a>
  <a href="#zoomOut" class="olControlZoomOut olButton">-</a>
</div>

After

The enhancement consists of replacing the anchor element with a button. The good thing of using a button is that it allows for additional markup inside the element, giving us room for a tooltip. The tooltip span element of the button is made visible on :hover and :focus using regular css, the tooltip also greatly enhances screen reader interoperability.

Modified and enhanced markup:

<div id="..." class="..."
    unselectable="on" style="...">
  <button class="olControlZoomIn olButton olHasTooltip_bttm_r">
    <span role="tooltip">Zoom in</span>
    +
  </button>
  <button class="olControlZoomOut olButton olHasTooltip_bttm_r">
    <span role="tooltip">Zoom uit</span>
    −
  </button>
</div>

These modifications were proposed in PR 1708 and PR 1249

LayerSwitcher

The LayerSwitcher suffers from some of the same problems as noted above, with the difference that the buttons to activate (expand / collapse) the control are actually div’s with images with an associated mouseclick handler. The bigger (compared to hyperlinks) problem with this is that these are only available to users with a pointing device.

Before

Markup of the original:

<div id="..." class="olControlLayerSwitcher..."
    unselectable="on" style="...">
  <div id="..." class="layersDiv" style="display: none;">
  <div class="baseLbl">Achtergrondkaart</div>
  <div class="baseLayersDiv">
    <input id="..." name="..." type="radio"
      value="0 buffer: OpenLayers WMS"
      checked="" class="olButton">
    <label class="labelSpan olButton"
      style="...">0 buffer: OpenLayers WMS</label>
    <br>
  </div>
  <div class="dataLbl" style="...">Overlays</div>
  <div class="dataLayersDiv">
  </div>
  </div>
  <div id="..." class="maximizeDiv olButton" style="...">
    <img id="..." class="..." src="../img/layer-switcher-maximize.png" style="...">
  </div>
  <div id="..." class="minimizeDiv olButton" style="...">
    <img id="..." class="..." src="../img/layer-switcher-minimize.png" style="...">
  </div>
</div>

After

Again the solution here is to use the proper semantic element for the expand and collapse actions; a button. Next to that, to enhance usability, the buttons are moved to the top of the control so they end up as first element in the tab order. The images were replaced with the ubiquitous hamburger symbol ≡ for opening and a multiply symbol × for collapsing the popover. Also extra focus handling was applied so that when expanding the control focus is moved to the first option that can be selected and when closing the control focus is moved back to the map (the map’s div element needs to be made programatically focusable by adding tabindex="-1"), next to that the selected option or checkbox remains selected until the LayerSwitcher is closed.

<div id="..." class="olControlLayerSwitcher..."
    unselectable="on" style="...">
  <button name="show" class="maximizeDiv olButton..." style="...">
    <span role="tooltip">Toon kaartlagen</span>
      ≡
  </button>
  <button name="hide" class="minimizeDiv olButton..." style="">
    <span role="tooltip">Verberg kaartlagen</span>
      ×
  </button>
  <div id="..." class="layersDiv" style="">
    <div class="baseLbl">Achtergrondkaart</div>
      <div id="baseLayersDiv" class="baseLayersDiv">
        <input id="..." name="..." type="radio"
            value="OpenStreetMap" checked="" class="olButton">
        <label class="labelSpan olButton"
            style="...">OpenStreetMap</label>
        <br>
      </div>
      <div class="dataLbl">Overlays</div>
        <div class="dataLayersDiv">
          <input id="..." name="POI" type="checkbox"
              value="POI" checked="" class="olButton">
          <label class="labelSpan olButton"
              style="...">POI</label>
          <br>
        </div>
    </div>
  </div>

This could be enhanced a little further by explicitly linking the label and the input elements using a for attribute or nesting the input elements inside their labeling element, however the above changes are enough to make the control keyboard operable.

The enhanced FramedCloud popup and KeyboardClick controls in action. Showing the tooltip on keyboard focus of the close button on the popup.

OverviewMap control

The OverviewMap is a widget that presents a small map with an indication of the bounding box of the parent map, the indicator may be moved using a pointing device thus doubling in function as a navigating tool as well. This control is using the same pattern as the LayerSwitcher.

Before

<div id="..." class="olControlOverviewMap..." unselectable="on" style="...">
  <div class="olControlOverviewMapElement">
    <div id="..." class="olMap" style="...">
      <div id="..." class="olMapViewport" style="...">
        <div id="..." style="...">
          <div id="..." dir="ltr" class="olLayerDiv olLayerGrid" style="...">
            <img class="olTileImage" src="..." style="...">
          </div>
        </div>
        <div class="olControlOverviewMapExtentRectangle" style="...">
        </div>
      </div>
    </div>
  </div>
  <div id="olControlOverviewMapMaximizeButton"
        class="olControlOverviewMapMaximizeButton olButton" style="...">
    <img id="olControlOverviewMapMaximizeButton_innerImage"
        class="olAlphaImg" src="../img/layer-switcher-maximize.png" style="...">
  </div>
  <div id="OpenLayers_Control_minimizeDiv"
        class="olControlOverviewMapMinimizeButton olButton" style="...">
    <img id="OpenLayers_Control_minimizeDiv_innerImage"
        class="olAlphaImg" src="../img/layer-switcher-minimize.png" style="...">
  </div>
</div>

After

I’ve replaced the images with proper buttons like the LayerSwitcher and added tooltips. The navigation function hasn’t been touched yet, my experience shows that even experienced users hardly know how to find this and is rarely used. Also since the map responds to the regular arrow keys on the keyboard, adding this function to the overview does not seem neccesary.

<div id="..." class="olOverviewMap..." unselectable="on" style="...">
  <div class="olOverviewMapElement">
    <div id="..." class="olMap" style="...">
      <div id="..." class="olMapViewport" style="...">
        <div id="..." style="...">
          <div id="..." dir="ltr" class="olLayerDiv olLayerGrid" style="...">
            <img class="olTileImage" src="..." style="...">
          </div>
        </div>
        <div class="olOverviewMapExtentRectangle" style="...">
        </div>
      </div>
    </div>
  </div>
  <button name="show" class="olOverviewMapMaximizeButton
        olOverviewMapButton olButton olHasTooltip" style="...">
    <span role="tooltip">Toon overzicht</span>
    +
  </button>
  <button name="hide" class="olOverviewMapMinimizeButton
        olOverviewMapButton olButton olHasTooltip">
    <span role="tooltip">Verberg overzicht</span>
    ×
  </button>
</div>

The information popup I tend to use most is the FramedCloud popup. It is the most feature rich of the available popups. The popup provides a dynamic panel floating over the map to show information to the user. Most commonly it will display attribute information of a feature in the map as a result of a user interaction with eg. a SelectFeature or a FeatureInfo control.

Markup-wise this control isn’t that interesting so I’ll spare you the code. The problem with the popup is that because it uses a clickable div with a background image it firstly may be hard to find and secondly it won’t react to key events, no focus, no click thus impossible to close without a pointing device. To fix it I inserted a tooltip enabled button as above and added a focus handler to move focus to the popup when it opens. When the popup is closed focus is moved back to the map.

To actually retrieve the attribute information in a keyboard friendly manner Eric Lemoine created the KeyboardClick control. This control enables moving a cursor on-screen using the arrow keys. With some additional code in the click handler it will select the first feature it finds closeby to the cursor.

onClick : function(geometry) {
  var lyrs = this.selectControl.layers, selTarget;
  var px = this.map.getPixelFromLonLat(
               new OpenLayers.LonLat(geometry.x, geometry.y)
             );
  // create a small polygon around the click
  px = px.add(this.handler.STEP_SIZE * 1.5, 0);
  var lonlat = this.map.getLonLatFromPixel(px);
  var radius = Math.round(lonlat.lon - geometry.x);
  var sides = 8;
  var rotation = 0;
  var clicked = OpenLayers.Geometry.Polygon
                  .createRegularPolygon(geometry, radius, sides, rotation);

  // hit detection, the first intersection is a hit
  for (var resized = 1; resized < 4; resized++) {
    // try a few (resized-1) times with larger click polygon each time
    clicked = clicked.resize(resized, geometry);
    for (var i = 0; i < lyrs.length; i++) {
      if (lyrs[i].getVisibility()) {
        for (var f = 0; f < lyrs[i].features.length; f++) {
          selTarget = lyrs[i].features[f];
          if (clicked.intersects(selTarget.geometry)) {
            this.selectControl.clickFeature(selTarget);
            return;
          }
        }
      }
    }
  }
},

When using WMS or WMTS layers this code can be limited to passing the coordinate pair of the click location to a feature info control as is done in the CBS viewer project.

For some of the other javascript and Sass code you can look into the openlayersmap DokuWiki plugin source tree.

Talk about this on twitter.

CBSviewer, an accessible, open standards based webmap viewer

2003-12-13T09:17:51-08:00

The past months I’ve been busy developing a web mapping application together with Statistics Netherlands to replace their current offerings of web mapping such as CBS in uw Buurt which have various accessibility issues and has Google looking in over your shoulder. The new application can be used with both the keyboard and a mouse and and will fallback to a non CSS and/or non Javascript version if needed (or requested) by the user providing a much better and safer experience.

Key features of the application are:

open standards based interfaces WMS, WMTS, OpenLS LUS and WCAG
easy modification using maven war overlay techniques (as an example you can look into the NOK viewer project on Github
easy to localize using property files / resource bundles for all the text
easy to configure thematic maps using a set of xml files
easy styling adjustments using Sass and modular CSS

The project builds on well documented, well supported and proven open source libraries such as GeoTools, OpenLayers, jQuery and jQuery UI.

We’re not quite there yet but you can follow the progress of the project through the Github site

Rich client screen showing WOZ (property tax) values aggregrated to 100m blocks

This post was previously published on GISpunt.

Performance and uptime monitoring Inspire View service

2003-12-13T09:17:51-08:00

The Inspire directive has some fairly strict requirements regarding performance and uptime of services (QOS) (see: 32009R0976 Annex 1 and the amendment. Monitoring these parameters can easily be done using a few Python scripts and RRDtool. This provides an environment that is both lightweight and portable across platforms.

RRDtool has been around for ages and is a de-facto instrument for lightweight logging systems gathering large amounts of data. The database aggregates the input providing various algorithms such as average, max/min value as well as way more complex methods. The aggregation makes it possible to use the same database for years while the file size stays constant and the amount of information just keeps growing. Current versions of RRDtool provide Python bindings out of the box, however I chose to use PyRRD because I was unsuccessful compiling them using Visual Studio 10 first time around.

I have chosen to monitor performance based on response time (the time needed for the initial byte to be received) and transfer time (the time needed for the last byte to be received).

Performance measurements, note that responses also vary because of variations in requested image size

To prevent caching of the image a pseudo random bounding box and image size are used so that each request is unique, this generates some variance in the response size so the amount of data transferred (total bytes) for each request is also logged.

Next to that uptime is monitored based on the correct mimetype of the GetMap response, the assumption here is that a mimetype other that requested means there was an error in the service, thus un-availability, this is a rather coarse approach, but it works for me because there is a separate error log that provides the details of a failure.

Any red line in this graph denotes a failure

I have a batch file that runs the probe script every five minutes and create a graph every fifteen minutes, a HTML page is used to display the resulting graph as wel as provide access to the logfile and the last request.

Browse or get the sourcecode.

This post was previously published on GISpunt.

GEOZET, Building a dual-mode GIS webapp

2003-12-13T09:17:51-08:00

Within the GEOZET viewer project a dual mode GIS web application is being developed by Geonovum as one of the launching products of the PDOK programme. Dual mode in this case being on the one hand a rich, map enabled client/GUI and on the other hand a lean non-javascript, non-css client/GUI for cases like screenreaders. Bart has written about the OpenLayers based “rich” client in his posts, I’m working on the “core” version, that this post is about.

Application

GEOZET viewer is a web application that provides access to government publications (government in this case the local, provincial and state governments and associations) through a geographic means. Publications are documents such as building, cutting, liquor and other permits and licenses, press releases, legislation and so forth. It has been commissioned by ICTU and will serve as the “launching application” of the PDOK motor project.

Architecture Overview

In a quick overview we have three open standards based webservices;

A tile service for the base map, only used in the rich client,
an OpenLS service for the Gazetteer/Geocoding that will probably run on the Adres Coordinaten Nederland database
and a WFS service that has geocoded metadata about the publications; this includes address data and a hyperlink to the publication.

The WFS service is specific for this application, the former two are part of the PDOK infrastucture. Because of some extra’s that we need (like returning area’s) we’ll probably be building our own Gazetteer using Hibernate Spatial and Lucene, more about this some other time.

Implementation

As part of this project I’ve been working on the WFS client that does the queries and renders the information based on user input. Two of the requirements to meet were easily transferable license(s) of the software stack between the hosting parties and platform independence. This boils down to using OpenSource toolkits on the Java platform. As most of the PDOK stack is based on Geoserver and Postgis already our choice was easy, GeoTools. It’s been quite a while since I’ve used GeoTools and a “first” look was quite overwhelming. I’ve opted to use the upcoming 2.7 release (which already has some milestones released) mainly because of the new, simplified Query and SimpleFeature objects that I need. This way implementing the WFS client, as a servlet, becomes a fairly straightforward exercise.

The servlet essentially receives query input from the user though a simple HTML form, using either POST or GET, a radius and a location and optionally some filter categories. The user doesn’t actually see the location coordinate pair, just the placename they’ve entered and which has been sent to the Gazetteer for lookup.

This information is used to create a CQL filter to retrieve information.

private Filter maakFilter(double xcoord, double ycoord, double straal,
		String[] categorieen) throws ServletException {

	Filter filter;
	final StringBuilder filterString = new StringBuilder();
	filterString.append("DWITHIN("
			+ this.schema.getGeometryDescriptor().getLocalName()
			+ ", POINT(" + xcoord + " " + ycoord + "), " + straal
			+ ", meters)");
	if (categorieen != null) {
		filterString.append(" AND (");
		for (int i = 0; i < categorieen.length; i++) {
			filterString.append(FILTER_CATEGORIE_NAAM + "='");
			filterString.append(categorieen[i]);
			filterString.append("'");
			if (i < categorieen.length - 1) {
				filterString.append(" OR ");
			}
		}
		filterString.append(")");
	}

	try {
		filter = CQL.toFilter(filterString.toString());
	} catch (final CQLException e) {
		throw new ServletException("CQL Fout in de query voor de WFS.", e);
	}
	return filter;
}

Once we have the filter constructed we can fire off a query to the WFS and the result is parsed and rendered in a HTML list to the client. This way the information becomes accessible to screenreaders and other types of small capability” devices. In the parsing of the response I also calculate the distance between the objects in the response and the requested place so that the list can be sorted based on this distance, I’m using the UserData Map on the feature to store this information and then the sort method from the Collections framework.

protected Vector<SimpleFeature> ophalenBekendmakingen(Filter filter,
		double xcoord, double ycoord) throws ServletException, IOException {
	final Query query = new Query();
	try {
		query.setCoordinateSystem(CRS.decode("EPSG:28992"));
		query.setTypeName(this.typeName);
		query.setFilter(filter);
		query.setPropertyNames(Query.ALL_NAMES);
		query.setHandle("GEOZET-handle");
	} catch (final NoSuchAuthorityCodeException e) {
		throw new ServletException(
				"De gevraagde CRS autoriteit is niet gevonden.", e);
	} catch (final FactoryException e) {
		throw new ServletException(
				"Gevraagde GeoTools factory voor CRS is niet gevonden.", e);
	}

	final SimpleFeatureCollection features = this.source.getFeatures(query);

	final Point p = this.geometryFactory.createPoint(new Coordinate(xcoord,
			ycoord));

	double afstand = -1d;
	final Vector<SimpleFeature> bekendmakingen = new Vector<SimpleFeature>();

	final SimpleFeatureIterator iterator = features.features();
	try {
		while (iterator.hasNext()) {
			final SimpleFeature feature = iterator.next();
			afstand = p.distance((Geometry) feature
					.getDefaultGeometryProperty().getValue());
			feature.getUserData().put(AFSTAND_NAAM, afstand);
			bekendmakingen.add(feature);
		}
	} finally {
		iterator.close();
	}
	Collections.sort(bekendmakingen, new AfstandComparator());
	return bekendmakingen;
}

In a later stage we’ll probably also provide a “REST like” url, I’m still not sure how to do this though as the URL should be something “human” readable/understandable; we might end up just supporting the location information and not the filter as that makes things much simpler.

This project will go live on overheid.nl and be open sourced on the Open Source Observatory and Repository (OSOR) November 1st, 2011.

This post was previously published on GISpunt.

Testing your map service with JMeter

2003-12-13T09:17:51-08:00

Apache JMeter is a powerful tool for load testing functional behaviour and performance of applications over the network. It is however not the most accessible or easy to use tool, mostly because of the overwhelming amount of options. Once started though there is no stopping! We use JMeter primarily to load test our GIS web applications (eg. eBRP) so that we can see if these scale to the sometimes huge amount of concurrent users, in these scenarios we test a web application as a whole, however as we move into a more service oriented architecture we feel a need to test at a smaller scale, ie. the service level. Since many of our map services are exposed with a REST endpoint this is the easiest place to attach our test to.

A common way of working with JMeter is to record one or more typical sessions using the JMeter built-in proxy and recording function, these are then edited so they can be parametrized with user credential and session information. When starting a recording session you must start off with a simple scenario, if you don’t you’ll just get lost in the amount of information that is recorded.

For testing our mapservice we can basically follow the same procedure; using the ArcGIS server built-in JavaScript viewer or a simple test application we can record a user session. The result is then (in this case) parametrized so we can look at random locations on the map and do identify operations at random locations., to initially go to a random location we have a list of 100 address locations (zip;number;x coordinate;y coordinate) in a .csv file that have been randomly drawn from our address database. Then we put in some elements called BSF PreProcessors that will do some simple calculus in JavaScript to determine bounding boxes and such as shown below.

// retrieve wkid and bboxSize
var wkid         = parseInt(vars.get('wkid'),10);
var bboxSize     = parseFloat(vars.get('bboxSize'));

// randomize zoomfactor, bboxSize
var randomnumber = 1+Math.random();
var zM           = 5*randomnumber*bboxSize;

// resize bbox
var xmin = parseFloat(vars.get('xmin')) + zM;
var xmax = parseFloat(vars.get('xmax')) - zM;
var ymin = parseFloat(vars.get('ymin')) + zM;
var ymax = parseFloat(vars.get('ymax')) - zM;

// store in memory
vars.put('xmin',xmin);
vars.put('xmax',xmax);
vars.put('ymin',ymin);
vars.put('ymax',ymax);
vars.put('bbox','{"xmin":' + xmin + ',"ymin":' + ymin + ',"xmax":' + xmax
              + ',"ymax":' + ymax + ',"spatialReference":{"wkid":' + wkid + '}}');
log.info('bbox updater zoom in: ' + vars.get('bbox'));

Next up is adding some expectations that we want to be fulfilled, eg. we want to have all responses a HTTP status code of 200 (which means OK), we want image request to have the proper mime type (ie. a png8 image should be returned with ContentType: image/png and we are expecting certain response times). These are set up by adding Response Assertions to our JMeter script.

Now we can add some advanced options such as Logic Controllers and Timers to further randomize the load and number of sessions to our service. And finally we want to have our results so we add in some Listeners such as graphs and tables. These Listeners can also be used to export the test results to an XML or CSV formatted file or an image.

Get going

So get started, download JMeter and load the script (.jmx) attached to this post. Get a list of point data in a csv file (the attached .csv is in Rijksdriehoek / EPSG:28992 which may or may not be useful to you), when you use your own you may need to tweak the format in the “CSV Data Set Config - adressen”, also you probably also need to tweak some of the JavaScript code in the preprocessors as these are based on a regular grid in meters. I guess if your data is in UTM you should be fine.

You need to specify/adapt the parameters in the test plan (users, iteraties, mapPath, gisHost, gisPort, wkid, fullExtent and zoomExtent) to suit your mapservice. To start off choose a small number for both users and iteraties such as 1. You should now be ready to go…

Download JMeter script .zip.

Interpret Results

JMeter really doesn’t know what you’re using it for, so interpreting results is sometime a challenge in itself. There are some Timeout parameters (assertions) that you can adjust to specify minimum thresholds for response times and there’s a large number of Listeners that will help you visualize the results or export them to something like CSV which you can then use in you favourite spreadsheet or reporting tool.

A sample Run

Using the posted .jmx I ran a test on one of our development test servers, these are virtualized dual cpu win2003 systems running on esx; they are not production grade systems. Below you can see some of the results of an optimized mapservice (.msd) containing an aerial photograph of the Netherlands stored in a 24x7 oracle10 database from our production environment. I have set up the testplan to use 50 users with 10 iterations each and a rampup time of 900 seconds using only the map requests (not the identify); JMeter unfortunately didn’t quite make it to the end of the test due to a memory errror, so this needs to be tweaked a bit in the startup script of JMeter; it’s likely the result of all the listeners in the .jmx so you may want to disable one or two of them.

Typical response result for a map image

In the results tree shown in the first screen capture I have only logged the errors; these are erroneous because the response took too long to load (more/longer than the mapRequestResponseTimeMillis set at 950 ms).

In the second screen capture you can see the aggregate results of the testrun, which clearly shows that it takes longer to get a more detailed part of the map from the database.

Aggregate reponse time graph

perfmon graph from the AGS server during the test

The test is incomplete without also monitoring the server. There are a number of built-in tools for windows, such as Perfmon and of course the taskmanager.In my set up I noticed a competition between the two SOC processes running the map and the Java process running the servlet engine that runs the REST interface, each of the managed to get up to 1/3 of the total available CPU.

CPU (green) and memory (yellow) utilization graph during test

ArcGIS Manager also provides some graphs showing throughput.

Throughput graph from the AGS manager console

This post was previously published on GISpunt.

GeoDienstenCentrum Blog

homepage

sitemap

DokuWikiSpatial: spatial indexing and search for your wiki

Indexing

Searching

DokuWikiSpatial: geotagging for your wiki

Geotagging

DokuWikiSpatial: maps for your wiki

Spatially enable your Wiki, part 1: mapping

About DokuWiki

Maps in your wiki

Enhancing OpenLayers control accessibility

Zoom control

Before

After

LayerSwitcher

Before

After

OverviewMap control

Before

After

FramedCloud popup and feature info control

CBSviewer, an accessible, open standards based webmap viewer

Performance and uptime monitoring Inspire View service

GEOZET, Building a dual-mode GIS webapp

Application

Architecture Overview

Implementation

Testing your map service with JMeter

Get going

Interpret Results

A sample Run