Not sorry for the inconvenience

Save all voicebox messages from your FRITZ!Box 7530 router

2023-12-28T00:00:00+00:00

My parents’ FRITZ!Box 7530 (FRITZ!OS version 7.57) voicebox had “filled up” and before clearing all messages I was looking for a way to download all of them as a backup.

There is no really convenient way to do this in the web UI (there were close to 200 messages, some dating back more than 4 years), so I resorted to JavaScript to print a list of curl commands to download each message as a .wav file.

Messages are listed in a table with the following format:

<tr class="thead">
    <th class="sortable"></th>
    <th class="sortable sort_by_date">Date<span class="sort_no">&nbsp;</span></th>
    <th class="sortable">Name/Number<span class="sort_no">&nbsp;</span></th>
    <th class="sortable">Your Number<span class="sort_no">&nbsp;</span></th>
    <th class="sortable">Duration<span class="sort_no">&nbsp;</span></th>
    <th></th>
</tr>
<tr>
    <td class="newicon" datalabel="28.12.23 16:00">&nbsp;</td>
    <td>28.12.23 16:00</td>
    <td datalabel="Name/Number">Caller's Name</td>
    <td datalabel="Your Number">1234</td>
    <td datalabel="Dauer">&lt; 1 Min</td>
    <td class="btncolumn" datalabel="">
        <button type="submit" class="icon fon_book" id="" name="" value="" title="" disabled=""><img src="/assets/icons/ic_fonbook_add.svg" alt=""></button>
        <a class="download icon" href="/cgi-bin/luacgi_notimeout?sid=<session uuid>&amp;script=%2Flua%2Fphoto.lua&amp;myabfile=%2Fdata%2Ftam%2Frec%2F<recording id>">
            <button type="submit" class="icon audio" id="play_1" name="play" value="1" title="Play message/Save">
                <img src="/assets/icons/ic_triangle_right_blue.svg" alt="Play message/Save">
            </button>
            <audio preload="auto"></audio>
        </a>
        <button type="submit" class="icon delete" id="delete_1" name="delete" value="1" title="Delete"></button>
    </td>
</tr>
...

We want to ignore the header row and for every subsequent row, get the date and the caller’s name or number, to build a meaningful file name for the to-be-downloaded .wav file, and the URL of the link. The link points to a script that emits the raw WAV audio as a binary stream.

This is a quick & dirty piece of JavaScript to paste into the JavaScript console of Chrome’s Developer tools (keyboard shortcut Ctrl-Shift-I) to generate a list of curl invocations to download each message with a useful file name:

console.log(Array.from(document.querySelectorAll('table#uiTamCalls tr:not(.thead)')).map(e => {
    return `curl -s -o "${e.childNodes[1].innerText} ${e.childNodes[2].innerText}.wav" '${e.querySelector('a').href}'`;
}).join('\n'));

My initial attempt was downloading the file via the browser by invoking .click() on each link, however that turned out to work less well as Chrome doesn’t really like downloading files over a non-secure connection, requires you to allow multiple simultaneous downloads via a popup box and even if you do, will only download 10 files at a time.

YMMV if you have a different FRITZ!Box model or FRITZ!OS version, however I’d expect a similar approach should still work.

Commands to replace the deprecated `apt-key` script

2021-01-01T00:00:00+00:00

apt-key is deprecated, however there is no replacement available, neither does the man page document how to replace the commands apt-key provides. Here is my attempt.

Note: gpg will by default create new keyrings in the (new) “GPG keybox database version 1”, whereas apt expects them in the (legacy) “PGP/GPG key public ring (v4)” format. Specify the prefix gnupg-ring: for the keyring file to make gpg use the legacy v4 format.

apt-key list

This commands lists all keys stored in /etc/apt/trusted.gpg and any .gpg or .asc files in /etc/apt/trusted.gpg.d.

for f in /etc/apt/trusted.gpg /etc/apt/trusted.gpg.d/*.{asc,gpg}; do
  gpg --list-keys --keyid-format short --no-default-keyring --keyring $f
done

apt-key adv

This command is used to download a key and store it in the “right” keyring. apt-key adv merges all keyrings into one, downloads the new key(s) and then merges back the changes. No need to replicate this setup.

Updating an expired key

If you’re updating an expired key, write it to the same keyring, replacing the expired key. To find any keyrings containing an expired key, run the following:

for f in /etc/apt/trusted.gpg /etc/apt/trusted.gpg.d/*.{asc,gpg}; do
  $(gpg --list-keys --no-default-keyring --keyring $f | fgrep -iq expired) && echo "Expired key in $f"
done

Once you’ve identified the keyring and key ID, download the new key:

sudo gpg --recv-keys --no-default-keyring --keyring=gnupg-ring:/etc/apt/trusted.gpg.d/<FILENAME>.gpg --keyserver keys.gnupg.net <KEY_ID>

Downloading a new key

When downloading a new key, create a new keyring in /etc/apt/trusted.gpg.d. Note that on recent versions of gpg, this keyring will be in “GPG keybox database version 1” format, which is incompatible with apt-key.

Choose a suitable filename for the new keyring and download the key:

sudo gpg --recv-keys --no-default-keyring --keyring=gnupg-ring:/etc/apt/trusted.gpg.d/<FILENAME>.kbx --keyserver keys.gnupg.net <KEY_ID>

How to create a transcript of a Google Support chat

2020-12-30T00:00:00+00:00

When you interact with Google Support via chat, you can ask for a transcript of the conversation to be sent to you. That transcript is in PDF format though. If that’s not suitable for you or you forgot to request a transcript, there is a way out.

Open the chat in its own pop-out window via the arrow icon on the right of the blue header bar. Then open the Chrome developer console (Ctrl-Shift-J), paste the following JavaScript code and hit enter to copy a transcript of the conversation to your clipboard.

const copyToClipboard = str => {
  const el = document.createElement('textarea');
  el.value = str;
  document.body.appendChild(el);
  el.select();
  document.execCommand('copy');
  document.body.removeChild(el);
};

copyToClipboard(Array.from(document.querySelectorAll('.chatsupport_cbf_qb')).map(e => {
  t = new Date(parseInt(e.getAttribute('ts'))).toISOString();
  if (e.querySelector('.systemMessageUserWrapper')) {
    return `[${t}] ${e.querySelector('.systemMessageUserWrapper').innerText}`;
  } else {
      n = e.querySelector('.chatsupport_cbf_rb').getAttribute('aria-label');
      m = e.querySelector('.chatsupport_cbf_ob').innerText;
      return `[${t}][${n}]: ${m}`;
  }
}).join('\n'));

How to upload your package to the Python Package Index (PyPI) test server

2020-05-23T00:00:00+00:00

Python packages are distributed via the Python Package Index (PyPI). The Python Packaging Guide provides details for uploading your project to PyPI. However the packaging guide is missing instructions for uploading to the PyPI test server. This is recommended as a trial run for uploading a new version of your package, since you upload a given version of your project only once. You can use the test server to e.g. verify your README renders correctly.

Assuming you have finished all the work on the new release of your project, written the release notes, increased the version number, tagged the release and are ready to publish, follow theses steps:

If you haven’t yet, create accounts on PyPI and the PyPI test server. Note that these accounts are entirely independent, however you may want to use the same username (but different passwords, of course).
For security reasons it is strongly recommended to create an API token instead of using your username and password when uploading a package to PyPI. If you haven’t done so, create an API token on both PyPI and the PyPI test server.

You can choose to restrict this token to only a single package, which you should definitely do if you use the API token e.g. in a CI/CD workflow. For your personal use I suggest you leave the token unrestricted, since there is no good workflow for switching between multiple API tokens.

Note that the API token will only be displayed once when you create it, so make sure you copy the token. If you forget to do that, revoke and create a new one.
Create a .pypirc file in your home directory to store your API tokens for authentication when uploading, with the following content:
```
[pypi]
username = __token__
password = pypi-AgEIcH...

[testpypi]
username = __token__
password = pypi-AgENdG...
```
If you haven’t done so, install twine: pip install --upgrade twine.
Create a source distribution and a wheel for your package:
```
python setup.py sdist bdist_wheel
```

Run twine check on your package files and ensure they pass:

$ twine check dist/*
Checking dist/dokuwikixmlrpc-2020.5.23-py2.py3-none-any.whl: PASSED
Checking dist/dokuwikixmlrpc-2020.5.23.tar.gz: PASSED

This command will report any problems rendering your README.

Upload your packages to the PyPI test server:

twine upload --repository testpypi dist/*

You should not be prompted for a username or password, since those are configure in your .pypirc. When successful, this should print the URL of your package on the test server:

Uploading distributions to https://test.pypi.org/legacy/
Uploading dokuwikixmlrpc-2020.5.23-py2.py3-none-any.whl
100%|████████████████████████████| 10.1k/10.1k [00:01<00:00, 5.23kB/s]
Uploading dokuwikixmlrpc-2020.5.23.tar.gz
100%|████████████████████████████| 9.63k/9.63k [00:01<00:00, 9.39kB/s]

View at:
https://test.pypi.org/project/dokuwikixmlrpc/2020.5.23/

Verify everything is as you expect e.g. there are no rendering errors.

Upload your packages to PyPI, fo realz:
```
twine upload dist/*
```
You should not be prompted for a username or password, since those are configure in your .pypirc. When successful, you should see output similar to the above.

Congratulations! You’ve done it! You package is now available to the world 🎉

Analysing Parliamentary Questions Answered at Accountability Hack 2015

2015-11-22T00:00:00+00:00

A very frosty November weekend marked the end of Parliament Week and the fifth anniversary of the Accountability Hack, originally named UK Parliament Hack, organised by Tracy Green from the Parliament Digital Service, Nick Halliday from the National Audit Office and Terry Makewell from the Office for National Statistics with very active support from the RebelUncut crew.

Hack #Parliament's data and show us what you can do with it, sign up to #AccHack15 https://t.co/mcN02u4NbV
— UK Parliament (@UKParliament) November 13, 2015

Hackers and “armchair auditors” were invited to tackle four different challenges using a diverse set of open data sources:

NAO: Use spend data and any other data set to improve accountability.
Parliament: Best use of linked data to improve accountability.
ONS: Use the ONS OpenAPI to improve accountability.
Wildcard: Use any three open data sets to improve accountability.

And we're LIVE! #AccHack15 is go pic.twitter.com/QHjV7QWOUd
— RebelUncut (@RebelUncut) November 21, 2015

Many prospective participants were deterred by either the freezing cold or by issues with public transport, so not that many heard Meg Hillier MP give the introductory address. After that, ideas were thrown around and teams started forming. I joined Natalia, Mina and Emma, a brilliant trio who were working on a visualisation of Parliamentary Questions Answered and were looking for some help with crunching the data and classifying the quality of answers.

Wonderful to have @Meg_HillierMP sharing her passion for holding "people like me" accountable #AccHack15 pic.twitter.com/q98Q85HkPq
— RebelUncut (@RebelUncut) November 21, 2015

We first of all needed to pull all data from the Parliament’s Linked Data API in JSON format. Downloading all 63000 questions in batches of 500 (which is the maximum batch size the API allows unfortunately) by hand was of course not an option, so I started by implementing a download script in Python. Pulling down all questions took several hours due to the rather poor performance of the API.

In the evening, Kevin ran a very entertaining round of the MLH !LIGHT challenge, where each contestant has 15 minutes to re-create a given website (in our case it was the bootstrap front page) using a very bare bones browser based editor with no syntax highlighting or auto completion. No navigating away from the tab to bring up help and you don’t get to see a rendered preview of your creation until after you submitted.

The overnight stay at the NAO was again quite comfortable and we could use a shower in the morning. My team from the Saturday did sadly not come back, however John Sandall, a good friend and brilliant data scientist, arrived in time for breakfast. We discussed classifying answer quality with an N-gram analysis using a list of previously identified phrases commonly used to defer questions. An alternative would be training a text analysis model on the entire text corpus based on a training set of manually classified answers.

@emjan29 @minaorangina @NataliaLKB I’m working with @frathgeber at #AccHack15 we have a big flat CSV of MPs Q&A, you coming back for today?
— John Sandall (@John_Sandall) November 22, 2015

I’m at Accountability Hack at the @NAOorguk today #AccHack15 using @ukparlidata #data to hold lawmakers to account! pic.twitter.com/JKCQgzN33C
— John Sandall (@John_Sandall) November 22, 2015

Before getting to work on that we needed to transform the raw data into a suitable form and identify which attributes were relevant for our analysis. Halfway through doing that I realised the answer text was missing from the data and found out it was due to passing a query parameter to the API (_view=all), which included extra fields, but left out the actual answer data. By that point it was unrealistic to be able to rerun the entire download in time for show & tell.

Need to pull all data from @UKParliData again, because ?_view=all contrary to what you might think does *not* give you everything #AccHack15
— Florian Rathgeber (@frathgeber) November 22, 2015

Full MPs answered questions #OpenData from @ukparlidata as 62k row CSV (7MB compressed) -> https://t.co/JmuqMdl3k9 #AccHack15
— John Sandall (@John_Sandall) November 22, 2015

John did however still manage to run some statistical analysis on the data to answer many interesting questions. Meanwhile I turned my download script into a “proper” Python package using PyScaffold, uploaded the package to PyPI and the documentation to Read the Docs - just in time for going up on stage!

Here we go!! @gabysslave kicking off #AccHack15 show & tell pic.twitter.com/kkTCNOZ5jG
— RebelUncut (@RebelUncut) November 22, 2015

Sounds like we're going to get some robust feedback on @UKParliData now... #acchack15
— Dan Barrett (@dasbarrett) November 22, 2015

Quite a few extra spectators came along to attend the show & tell with a rather impressive lineup of 16 projects! I was on stage twice: First to present the results of our analysis of “Any Questions Answered?”, which had revealed some interesting insights. Jim Shannon MP asked the most questions, the Department of Health had to answer the most. The Foreign & Commonwealth Office was the slowest to respond. Nick Clegg’s questions were ignored the longest and the Prime Minister referred the highest proportion of questions. With more time to build up a training set by categorising some questions manually e.g. for quality or difficulty, we could have trained a Bayes classifier for the entire corpus.

#acchack15 next up any questions answered... pic.twitter.com/4KaEJHr64m
— Ranjan Balakumaran (@financialeyes) November 22, 2015

Fun fact: @nick_clegg ranks higher than any other MP for how long it takes to get questions answered (49 days on avg) #PoorNick #AccHack15
— John Sandall (@John_Sandall) November 22, 2015

"Have all these MPs always been so annoying?!" @John_Sandall Love it #AccHack15
— RebelUncut (@RebelUncut) November 22, 2015

Later I went on stage again to present DDPy, a command line interface to interact with the Parliament Linked Data API, which had evolved out of my download script to pull the Parliamentary Questions Answered. I decided to solve this once and for all and, wrote a generic downloader in Python and put it on PyPI. Now anyone can easily download any data set after a simple

pip install ddpkuk

Ok so *now* @frathgeber is going to give us some robust feedback on @ukparlidata #acchack15
— Dan Barrett (@dasbarrett) November 22, 2015

DDPy from @frathgeber #AccHack15 https://t.co/YeBULBhRrt
— RebelUncut (@RebelUncut) November 22, 2015

Read how to use DDPy to access @UKParliData #data at https://t.co/iJTmrdEv89 and get the code on @GitHub https://t.co/ufLZdFP0pk #AccHack15
— Florian Rathgeber (@frathgeber) November 22, 2015

The judges apparently came away quite impressed with both our presentations since we received an honourable mention for the “Best Analysis of Parliamentary Data” for Any Questions Answered? and the “Best Tool for the Community” for DDPy. As if that wasn’t enough, I was quite touched for also being awarded a “Community Spirit Prize”. It was (and continues to be) an honour and pleasure serving the community!

Got an honourable mention at #AccHack15 for "Any Question Answered?" our analysis of Parliamentary Questions Answered /w @John_Sandall
— Florian Rathgeber (@frathgeber) November 22, 2015

Got another honourable mention at #AccHack15 for DDPy as a useful tool for the #hackathon community. So go & use it! https://t.co/80K4puh9Wh
— Florian Rathgeber (@frathgeber) November 22, 2015

No end of awards today it seems ;) Many thanks to @RebelUncut @nickmhalliday and @greentrac for running #AccHack15 https://t.co/9hvSrkubyY
— Florian Rathgeber (@frathgeber) November 22, 2015

Any Questions Answered?

Code
Slides

DDPy - data.parliament.uk for Humans

Other resources

Accountability Hack 2015 hackpad

Chilterns circular | Reading - Nuffield - Chinnor - West Wycombe - Marlow - Reading

2015-09-06T00:00:00+00:00

With a warm sunny day forecast for Sunday - potentially the last summer day - it was another perfect opportunity for a longer ride. Steffen had suggested a route through the Chilterns, which promised lots of ups and downs, and brought his vintage road bike. The weather did indeed not let us down and it was a great day out with a rewarding if not exhausting ride of over 90km. Unfortunately the longest downhill stretch was a very poor road littered with potholes so we had to be very cautious and couldn’t nearly go full speed, which was a slight downer.

For our slightly-more-than-halfway stop in West Wycombe we were rewarded with a great view from a hill that hosts the Church of St. Lawrence and the Dashwood Mausoleum. The mausoleum is a very fascinating and peculiar construction, which seems somewhat out of place.

Church of St. Lawrence

Dashwood Mausoleum

We gave the “Hellfire Caves” a miss, given that their lurid name indicated it was either a family affair or a tourist trap. The admission was rather pricey and we only had sunglasses, which aren’t exactly helping in a cave…

Hellfire Caves

Later we passed the rather picturesque riverside town of Marlow, which seems like a prettier and quieter version of the somewhat overly hyped Henley.

All Saint’s Church Marlow

The Compleat Angler

How to speed up your vim startup time

2015-07-31T00:00:00+00:00

My go-to editor for pretty much anything is vim. And like any serious vim user I use numerous plugins to extend and improve the editing experience. Natually there’s a cost to that, namely an increase in vim startup time. Compared to almost any GUI application vim starts up almost instantaneously. Still I noticed that slight delay and it had been bothering me for a while until I finally got round to investigate and tweak.

Thankfully vim makes profiling the startup time really convenient by providing a --startuptime flag to write timings for loading your .vimrc and plugins to a file, which looks something like this:

times in msec
 clock   self+sourced   self:  sourced script
 clock   elapsed:              other lines

007  000.007: --- VIM STARTING ---
399  002.392: Allocated generic buffers
468  000.069: locale set
946  002.478: GUI prepared
954  000.008: clipboard setup
967  000.013: window checked
181  011.214: inits 1
187  000.006: parsing arguments
188  000.001: expanding arguments
212  000.024: shell init
230  001.018: Termcap init
203  000.973: inits 2
390  000.187: init highlight
090  002.745  002.745: sourcing /usr/share/vim/vim74/debian.vim
679  000.296  000.296: sourcing /usr/share/vim/vim74/syntax/syncolor.vim
805  001.004  000.708: sourcing /usr/share/vim/vim74/syntax/synload.vim
...

The interesting measurements are those concerned with sourcing files, so focus your attention on the 3rd column to see where time is spent. Short of trimming down your vimrc (mine takes about 18ms to load by itself) and cutting down on the number of plugins, the plugin manager can also make a difference. I used pathogen for a long time, before switching to Vundle (because it’s easier to have your plugin manager handle Git repositories rather than having to manually add them as submodules to your dotfile repository or similar). Then I came across vim-plug and was intrigued by its on-demand loading feature. That allows loading plugins for specific file types or only on the first invocation of a certain command. The latter is particularly useful for plugins you don’t need to have active all the time. In my case this was particularly useful for NERDtree and DokuVimKi, which both take a significant time to load. Looking carefully at the startup time output I also noticed I was sourcing filetype plugins twice! The offending line in my vimrc was quickly found, which shaved off some additional milliseconds.

TL;DR Switching to vim-plug, getting rid of some unnecessary plugins, loading others only on demand and uncluttering my vimrc I managed to cut down my vim startup by more than half, from close to 250ms to only about 120ms.

Plotting Christmas gift tags using a Raspberry Pi and resin.io

2014-12-23T00:00:00+00:00

On the first weekend of December, the fine folks from resin.io put on an Xmas IoT Hackathon at the London Fab Lab. The Fab Lab is a maker space for digital fabrication and rapid prototyping. They host classes and events (like hackathons) and members can use a variety of tools for wood and metal working as well as 3D printers and a laser cutter, which you can use e.g. to cut a gingerbread house like this one (made from real gingerbread!):

Appropriate to the occassion, the resin folks brought boxes full of hardware:

There was therefore no shortage of Raspberry Pis and all kinds of accessoires. Teams were formed and a range of exciting hacks started taking shape. I teamed up with Mark, a good friend and hackathon buddy, who had brought along a Graphtec Silhouette Portrait cutter/plotter which we were intending to hack:

You’d normally use this to cut out shapes previously created with their proprietary software Silhouette Studio. Of course that’s only the intended use. Replacing the knife by a silver sharpie turns the Silhouette into an electronic circuit plotter - for very simple circuits. Here’s a proof-of-concept circuit used to power an LCD with a button cell:

Our plan was to the Silhouette into a Christmas gift tag plotter, powered by a Raspberry Pi. Using Silhouette Studio to drive the plotter was of course no option. We had to find a pure command line, open source solution to generate the template and feed it to the plotter which also needed to run on the ARM processor of the Pi. After some searching and experimenting we found a Python based driver, which after a little tweaking was able to digest a postscript file and get it plotted. Here are some experimental results:

We generated the gift tags using paper.js: The user puts in five names of friends to create tags for and we place the names, framed each by two automatically generated snowflakes, on a canvas and have paper.js save the canvas as SVG. Now there were only two pieces missing: one was converting from svg to postscript, which inkscape happily does for us, even on the command line. The final piece was then a simple node.js server to put everything together: A form to put the names in, which when submitted generates an SVG string, which is POSTed to the server, where it is saved to a file, converted to postscript via inkscape and then fed to the plotter. Job done! Well, almost, we still had to deploy it to the Pi.

Fortunately resin.io, a deployment service for IoT devices, makes that rather simple: you create an application on the resin.io dashboard, download the base image for your device, flash it on the SD card, pop it into the Pi and boot it. After a few minutes the device comes online at the dashboard. When you then push to the Git repository associated with the project, your new code is deployed on the Pi, heroku-style, but cooler (because of the blue unicorn and cause it run on your device!).

The device is provisioned via docker and resin.io provide a base container (based on Raspbian wheezy) that comes with node.js preinstalled. Python, git and inkscape are easily installed via the apt package manager and we simply git clone the graphtec driver. The container automatically runs npm install and when successful npm start to run our server. So far, so straightforward, but there were two small issues to figure out: one was installing the font we were using for the names and refreshing the font cache and the other was detecting the printer, which required starting udevd manually. Having that sorted, our first test was finally sucessful:

We had a really great weekend at the Fab Lab and even made the 2nd prize with our hack. Thanks to the resin.io folks for putting it on and hopefully until the next one!

How to update expired repository keys in Debian / Ubuntu

2014-07-24T00:00:00+00:00

When using third party package repositories, you occassionally might need to update expired repository keys. An expired key leads to an error message during a sudo apt-get update similar to the following:

W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used.
GPG error: http://build.i3wm.org raring Release:
The following signatures were invalid: KEYEXPIRED 1396011159 KEYEXPIRED 1396011159 KEYEXPIRED 1396011159

To find any expired repository keys and their IDs, use apt-key as follows:

apt-key list | grep expired

You will get a result similar to the following:

pub   4096R/BE1DB1F1 2011-03-29 [expired: 2014-03-28]

The key ID is the bit after the / i.e. BE1DB1F1 in this case.

To update the key, run

sudo apt-key adv --recv-keys --keyserver keys.gnupg.net BE1DB1F1

The repository will then be updated with the next sudo apt-get update.

Taarifa reporting for International Medical Corps UK at hack4good London

2013-10-06T00:00:00+00:00

On the weekend of 4-6th October I was part of #hack4good in London working with Chris Adams, Corinne Pritchard and Nick Stanton tackling multi-agency reporting at refugee camps for International Medical Corps UK with the Taarifa platform.

We approached the problem from two angles: One was developing a workflow for a paper-based reporting system, where report forms are automatically generated for a specific area like a refugee camp. Refugees would be able make a report by filling in the form themselves with the help of a simple iconography to overcome the language barrier, mark the location on a map and drop it into a collection box. The forms are machine readable and can be digitized in an automated process.

The other was the technical challenge of providing a platform that could process a variety of different reports that various agencies working at a refugee camp need to deal with from santiation over general feedback to sexual exploitation. These can be aggregated in one place by the Taarifa API which I’ve extended to be able to handle any report schema. Creating a new report type is a simple as sending a POST request describing the schema to the API in JSON format. Reports can then be submitted immediately via the JSON API or via an automatically generated web form. All of that is live and operational at http://api.taarifa.org.

There was quite a bit of media coverage around #hack4good: Channel 4, SkyNews and a video of the London event by Big Bang Lab.

C pointers to multi-dimensional assumed shape Fortran arrays with gfortran

2013-08-12T00:00:00+00:00

Interoperability between C and Fortran codes is often complicated by Fortran’s support for “true” multi-dimensional arrays. In addition, Fortran 90 supports assumed shape (also referred to as deferred shape) arrays, where the extent sizes are determined at runtime. Actual arguments for array dummy arguments can be array slices that may be of non-unit stride. In short, an assumed size Fortran array can reference non-contiguous chunks of memory.

Therefore, the compiler will reject an attempt to obtain a raw data pointer of a Fortran array to pass to a C function like in the following module:

gfortran 4.7 emits the following error message:

Error: Assumed-shape array 'v' at (1) cannot be an argument to the procedure 'c_loc' because it is not C interoperable

What if you know that the assumed-shaped array passed to your function is actually contiguous (or you choose to leave it the caller’s responsibility)? Surely the compiler should be able to give you a pointer to the first array element like so:

Turns out gfortran doesn’t and generates the same error as before. The gfortran developers have recognized this as a bug, which will be fixed in the 4.9 release.

In the meantime you can work around it by defining a pure function that simply returns the first argument of an assumed shape array. gfortran is happy with that since now the type of the argument passed to c_loc is a scalar and therefore interoperable:

However, this workaround confuses Intel’s and PGI’s Fortran compilers, so simply escape it using the a preprocessor #define: