ground- water, geo- statistics, environmental- engineering, earth- science

Digging Into My Research Database

without comments

A new version of Script Debugger was released recently, and I dug a bit into it, using my research database Papers.

For fun, I linked AppleScript (that digs into my database on MacOS) with Python, that processes the data (creates a histogram).

The process worked nicely, and being able to debug AppleScript is wonderful.

More info at

Written by Claus

March 6th, 2018 at 9:22 am

Posted in

Smartphones and Creative Ideas

without comments

At NASA, at least some people get rid of “smart phones” to get creative ideas back: Lynda Barry at NASA’s Goddard Space Flight Centre (via

Barry’s impact on the assembled Goddard employees was immediate; from the moment she arrived, she insisted on abandoning all electronic devices. “They were really flipped out about it,” says Barry. “The phone gives us a lot but it takes away three key elements of discovery: loneliness, uncertainty and boredom. Those have always been where creative ideas come from.”

At the time of writing this, the Süddeutsche Zeitung insists that social media (WhatsApp) “belong into classrooms

update 2017-Oct-11

  • die Tagesschau reports that 14-29 year old Germans are online for about 4.5 hours per day
  • the guardian has a longer report on how smartphones are hijacking ones minds. The text warns about a much more severe consequence: “Drawing a straight line between addiction to social media and political earthquakes like Brexit and the rise of Donald Trump, they contend that digital forces have completely upended the political system and, left unchecked, could even render democracy as we know it obsolete.” The article goes on to explain how there are certain hooks emplaced in smartphone-related technology that are designed to keep you there and make for the companies advertising dollars.

Written by Claus

September 13th, 2017 at 10:56 am

Posted in

Days 2&3 at #spatialstatistics2017

without comments

It became increasingly difficult to post updates on the spatial statistics conference. The icebreaker, another day full with diverse interesting talks, the dinner, another day that ended the conference with an interesting session honouring the achievements of Peter Diggle. Former and current colleagues such as Paulo Ribeiro and Emanuel Giorgi gave enlightening talks that stressed both the scientific achievements and the great kindness and humanity of Peter Diggle. CHICAS, the center for health informatics, computing, and statistics, is the current culmination of his efforts.

It’s hard to pick topics that stood out during the last two days of the conference, just because there were many great talks on a large variety of topics. Here is an attempt.

Point Processes

There were a number of talks covering Point Processes, notably the keynotes by Thordis Thorarinsdottir and Rasmus Waagepetersen. Thordis had a variety of interesting quotes including this one by Frank H Bigelow from 1905:

There are three processes that are generally essential for the complete development of any branch of science, and they must be accurately applied before the subject can be considered to be satisfactorily explained. The first is the discovery of a mathematical analysis, the second is the discussion of numerous observations, and the third is a correct application of the mathematics to the observations, including a demonstration that these are in agreement.

Thordis urged the need for more and better inference methods. I might be worth pointing out that Bigelow went on to state that

Often a good theory is misapplied to good observations, or good observations are explained by a poor theory.

In summary, these thoughts are not too far away from Peter Diggle’s triangle, pictured above.


There were two nice talks that employed copulas for multivariate spatial models and one that I missed, unfortunately:

  • Jonathan Tawn from the University of Lancaster presented on “Modelling Spatial Extreme Events“; he takes great care of marginal distributions and how to reasonably include extremes there for a better joint representation in copula space;
  • Fakhereh Alidoost and Alfred Stein from the University of Twente presented on “Interpolation of Daily Mean Air Temperature Data via Spatial and Non-Spatial Copulas
  • the talk that I missed was entitled “Hierarchical Copula Regression Models for Areal Data” presented by D. Musgrove, J. Hughes and L. Eberly


Written by Claus

July 12th, 2017 at 3:48 pm

Posted in

Day 1 at #spatialstatistics2017

without comments

Peter Atkinson opened the conference with pointing out the broad scope of the conference: “one health” (e.g., CDC, UC Davis) that relates to human, veterinary, and environmental health. I was glad that my talk with interpolating groundwater quality data fit right into that scope.

I saw too many interesting talks and met too many interesting and nice people, to list everything here. Instead, this is a small selection.


First off, it’s nice to encounter similarly minded work. Particularly, I was happy to see the following presentations:

  • Emilie Chautru presented a poster entitled “Cokriging of Nonnegative Data on the L1 Sphere”, on Cokriging compositional data;
  • Svenia Behm from the University of Passau presented a talk entitled “Statistical Inference in the RIO Model – the Detrending Step Revisited”. She calculates something similar to my “locally mixed distributions”;
  • A. Lawson pointed out the importance of properly taking censored measurements and true zeros into account, both in his keynote (“One Health: Spatial Statistics at the Border of Human and Veterinary Health”) and in his talk (“Bayesian Cure-Rate Survival Model With Spatially Structured Censoring”). I didn’t talk about it at this conference, but it is dear to my heart;

Cool Stuff

  • M. Pereira showed cool images of road crash density estimates based on data from Paris, France. Benedikt Gräler showed a poster with the Envirocar initiative. Data related to driving patterns and fuel consumption is collected while driving, is analysed, and can be viewed online.
  • Samir Bhatt gave a great keynote presentationon mapping malaria endemicity. Besides the interesting issues related directly to malaria, this talk raised some interesting questions on modelling philosophies. Samir Bhatt proposed “richer models” as a way forward beyond his current practice of using multivariate models. Alternatively, he phrased it as models that “include mechanisms”. Peter Diggle asked how his approach relates to the concept of parsimonity. It is interesting to me that Samir Batt suggests to include mechanistic models in his data driven models, whereas for the groundwater quality mapping project I am working on, I have moved to a stochastic model. On the scale of the state, I see that deterministic, pde-based models are not feasible (too many unknown parameters and processes).

Written by Claus

July 6th, 2017 at 8:44 am

Posted in

New Papers!

without comments

I published two new papers recently! Find the titles and the links to more information below. Happy reading!

  1. Detecting and Modelling Structures on the Micro and the Macro Scales: Assessing Their Effects on Solute Transport Behaviour” – This paper sheds light onto a tricky issue: Is a spatial data-set stationary or not? This paper shows a method that can help to decide to delineate a boundary (“macroscale”) between regions that are at least somewhat more stationary than the entire domain. Furthermore, this paper
    • validates the algorithm based on a data-set where a boundary layer has previously been delineated;
    • demonstrates the effects of the macro structure and the smaller scale heterogeneity (“micro structure”) on solute transport behaviour; The micro structure is modelled by multivariate Gaussian and multivariate non-Gaussian structures.
  2. Estimating a Representative Value and Proportion of True Zeros for Censored Analytical Data with Applications to Contaminated Site Assessment” – True zeros such as no precipitation occur frequently in nature. This is one of the very few studies I know that treats those values statistically meaningfully and is based on a real-world data-set. We applied the methodology on a data-set related to contaminated sites, but this has implications everywhere else.

Written by Claus

June 29th, 2017 at 9:03 pm

Posted in

Own Your Writing

without comments

I just posted on about “Own Your Writing!”.

In this post, I

  • discuss how important it is to own what you write, even if it comes at a cost: Knowing technology and money. Also, it seems like publishing has become more complicated than it needs to be on open solutions.
  • play with the new JSON-feed format (in python)
  • Written by Claus

    June 21st, 2017 at 9:48 am

    Posted in


    without comments

    In my work about spatial dependence, I do see that in different ranges of quantiles, the type of dependence can differ. More generally, this means that thresholds are an important characteristic of environmental systems.

    This is why I think this video that I noticed on is so inspiring: sometimes something small leads to a big change — a “threshold” is “jumped over”:

    Written by Claus

    June 7th, 2017 at 3:37 pm

    Posted in

    Years of Blogging

    without comments

    • Manton Reece celebrated 15 years of blogging in March. It turns out that does not date that far back, but almost 11 years of blogging is not nothing either. The details are actually blurry in my mind. Definitely Dayf had started the “boardinger” at some time around 2003, we had used that for a while, and I guess I moved to WordPress in 2006
    • In fact, the WordPress installation at sysprovide has been running and has been upgraded constantly ever since – until last Thursday, when the first outage occurred (at least to my knowledge)
    • If there is one thing certain, it’s that things do change. As does the blogging frequency. I guess there are certain cycles
    • Recently, I started to take as my professional site more seriously, and setup a static page generator (Pelican) that supports Jupyter notebooks. While that is pretty exciting, I do not want to neglect planet water! This is the first post that I am writing and posting with Ulysses and not with my traditional TextMate / MarsEdit combination

    Yay to Independent Blogging!

    I will use the occasion of looking back, following Gabe Weatherhead’s spirit, to give a shout out to independent blogging.

    The following three blogs are in my queue since the beginning of time, none of which are particularly related to water:

    I also admit that I follow a few that are on Gabe’s list, and Gabs himself:

    Written by Claus

    April 9th, 2017 at 8:46 pm

    Posted in

    Update on “Learning and Playing” Update on “Learning and Playing”

    without comments

    Over a year ago I wrote a post on “Learning and Playing“. In the meantime, three important things happened, that lead me to update the original post:

    • yesterday, Apple announced the release of “Swift Playgrounds
      • this is the review of Rene Ritchie at iMore. He says: “It’s one of the finest things Apple has ever done, and it’s going to change the way coding is done for the next generation.”
    • Lorena A. Barba published a blog post on “Computational Thinking
    • “s/buy/make/” published a wonderful empirical statistical analysis that demonstrates how the complexity of legos is increasing

    An old note for my original post included this note:

    From a big picture view, it seems to me that it is more easy now than it was at the time Mindstorm was published to access computer programming. In contrast to this development (now there are more high level programming languages, now there are more simple plotting APIs) most of the (young) people today regard “the computer” more of a consuming and communication device than an invention and try things out device. More and more people use computers, but at least relatively less people use it for creation – and I think creation involves some way of programming.

    I expect that Swift Playgrounds, once it’s released, will offer the best platform since the original Logo, to learn and play. This is the announcement that made me the most happy among all recent Apple announcements. Lorena A. Barba reframed my “learn and play” phrase into “the essence is what we can do while interacting with computers, as extensions of our mind, to create and discover“. I expect Swift Playgrounds to be a wonderful tool for just that.

    Swift playground
    Screenshot of the Swift Playground Demo site.

    Let’s end with some good news: Lego is holding still onto its original values:

    So what happened with Legos? They made a lot more of them. In doing so, they made a lot of new, specialized bricks, but they made even more general purpose bricks. This trend is easily obscured by the opposite trend in the number of brick types, but from a ‘creative play’ standpoint the bricks you actually end up with are more important than the bricks you could have ended up with.

    And: “yeah… you can do this! Ba… Ba.. Baaa…!” (watch until the end!)

    Written by Claus

    June 14th, 2016 at 11:49 am

    Posted in

    automation: getting papers into papersapp from OmniFocus tasks

    without comments

    I am using OmniFocus to organize my life and in an attempt to get a few things done. My academic live also involves staying on top of publications. I rely on papers as my reference manager. In the last little while, I used OmniFocus to keep track of the papers that I found and that I wanted to get and / or read. This has been a quite reliable workflow, but involved quite some manual clicking.

    OF papers

    This post describes how I improved it with a little applescript to go more directly from OmniFocus to papers:

    • whenever I see a paper that interests me, I capture it into OmniFocus;
      • the title of the task is the title of the paper or whatever else helps me to identify the paper;
      • the note contains the URL to the pdf of the paper, and nothing else;
      • the task gets assigned one particular project, whose sole purpose is to collect papers I want to get / download;
      • the capturing works mostly via Omni’s “clipotron“, the share sheet extension in iOS, or from within Reeder;
    • regularly, I check that project (on its review date) and get the papers. Until very recently, this involved a few steps for each paper: open the note in OF, click on the link, the relevant page would open in Safari, and I would klick on the bookmark tool that I had setup such that this webpage would open in papers. I replaced this with a very simple applescript (see code listing below)
      • make sure that I am connected to the University’s network to ensure that most papers are accessible;
      • select the perspective that focusses on the project that contains the papers I want to get;
      • select the papers that I want to get, hit a keyboard shortcut associated to the applescript in Keyboard Maestro;
      • tada! papers opens the links, tries to retrieve the pdf (which works in most of the cases) and the bibliographic information. All is left for me is to add some meta-data and read;

    Below you can find a listing of the applescript code. It is fairly simple and contains only a few lines. I can see a few areas where it could be expanded (parse URL from note if it contains more stuff than just the URL; what if pdf and / or bibliographic information can not be retrieved by papers). But for most of my use cases it works remarkably well. Hence, I would well agree to John D. Cook’s line of thought that yes, it’s a bit about time being saved, and it’s also about not being derailed. It’s also about accuracy (as followed up by Dr. Drang), and about knowledge transfer and improved processes, as Mike Croucher points out.

    Here’s the code snippet:

    tell application "OmniFocus"
        -- Target the content of the front window
        tell content of front window

        -- get selected entries
        set theTasks to value of every selected tree
        -- loop over each selected task
        repeat with aTask in theTasks
            tell aTask
                -- extract the task name and the note
                -- note contains URL
                set theTaskName to name
                set theNote to note
                display dialog theNote
                -- open in papers; it automatically retrieves the pdf and 
                ---    the bibliographic information (mostly)
                tell application "Papers"
                    open location theNote
                end tell
            end tell
        end repeat
    end tell

    end tell

    Relevant links regarding applescript with the two main software packages used:

    Written by Claus

    December 30th, 2015 at 10:39 am

    Posted in