planetwater

ground- water, geo- statistics, environmental- engineering, earth- science

Archive for 2013

identi.ca updates

without comments

Written by Claus

June 6th, 2013 at 10:30 pm

Posted in identi.ca

Floods in Germany

without comments

It has been raining quite a bit over the last weekend. The German Weather Service is reporting on its homepage at some stations (Aschau-Stein, Kreuth-Glashütte) ~400L per square meters within 90hours (3.75 days). As a comparison, the annual precipitation sum for Cologne is ~800L per square metre.

This has led to some extreme water levels, for example in Passau:

NewImage

The magic water level for the city of Passau is 12,55m, which has been exceeded for the first time since the year 1501AD (don’t ask me what this really means, since most likely there has been some change in the river’s regime over the years…)

The comparison pre-flood and currently looks quite impressive:

NewImage

Generally, the East and the South is affected. For example, a major highway close to lake Chiemsee is closed for traffic. Some people have kept their sense of humour, surprisingly:

NewImage

 

 

update Tuesday; June 04, 2013:

Prof. Harald Kunstmann gave an Interview during a TV special on the current flooding on German TV last night — this is the URL (starting around minute 31)

 

 

Here is a comparison of the gauged water height at the station Passau in the Danube. The white bars occur when there was no measurement due to the destruction of the gauge. Since this accident happened, the responsible agency is “taking individual measurements” according to their website. Also interesting, the actual peak water hight was higher and later than predicted 24 hours before.

Comparison WaterGauge

 

 

 

Written by Claus

June 3rd, 2013 at 1:44 pm

Posted in

identi.ca updates

without comments

Written by Claus

May 30th, 2013 at 10:31 pm

Posted in identi.ca

Improved Individual Decision-Making: Weather

without comments

When I read Rufus Pollock’s editorial on “Forget big data, small data is the real revolution”, it occurred to me that everybody, probably even I, could take advantage of what Pollock calls the “democratization of the masses”. In this post I will show how information can be “pulled together” using only basic programming skills. This information then can be used for improved decision making. The example that I decided to use to put this into practice might be the most universal conversation topic: weather 😉

Practicing “Small Data”

Usually, I follow my interest in weather on a very basic level: I read the weather forecast. I try to use not the basic forecasts. Hence, I like to visit wetterzentrale.de because of the fantastic amount of information they make available, and the fantastic visualizations that forecast.io and forecast.io/lines present.

The unfortunate thing is that you kind of have to believe those products. I hadn’t seen a good weather map in a long time, until I was sailing recently at DHH Chiemsee, who make prints from the DWD analysis maps of air pressure at ground surface (together with annotations of observations) available on a daily basis.

The following ideas came to my mind:

  • it would be very interesting to see the progression of these pressure maps over time
  • since they are analysis maps, commonly still hand drawn, it would be interesting to compare them to other analyses, done by somebody else
  • a description of the current situation associated with the pressure maps would be useful, so that an amateur like me gets some hints

With this information at hand, everybody could form their own opinion of the current weather situation in an improved way!

After some research, that did not take very long at all, I found some other sources on the internet, that allowed me to come up with the following map:

Grosswetterlage overview 2013 05 07 07 30 08

The code I wrote allows to create this plot at times that can be specified. The left column shows the current analysis performed by different institutions, the right column shows predictions performed by KMNI.

I wanted to do all this in python, so I needed to figure out how to get images from the internet and learned about the packages urllib2, HTMLParser, Image (I didn’t know that there was a greyscale png), and sched. Despite the fact, that there are still some (minor?) things that need to be ironed out (plotting of text with matplotlib, style of the headings) I put the code up on github.

I’d be very happy to hear what you guys think! Happy birthday Ferdi! 😉

Written by Claus

May 11th, 2013 at 10:32 pm

Posted in

identi.ca updates

without comments

Written by Claus

May 9th, 2013 at 10:30 pm

Posted in identi.ca

More than half of the world’s population lives inside this circle

without comments

NewImage

 

… and there’s a lot of water in that circle too…

via Very Spatial

Written by Claus

May 7th, 2013 at 9:53 pm

Posted in

Twitter Digests

without comments

Folks,

I just experimented with twitter digests on this blog —  a feature that has been broken since twitter did some changes to their api. I am suspecting that this might have lead / might lead to some blog posts showing up in your RSS feed, which are actually “just” twitter posts. Sorry for that inconvenience.

Written by Claus

May 6th, 2013 at 9:48 am

Posted in

Stimulating Presentation on Supposedly Boring Topic: Data Types

without comments

This is awesome, funny, shocking, and horrifying — all at the same time and for the entire four minutes! Quick demonstration about types in ruby and java script

via Hillary Mason

Written by Claus

May 1st, 2013 at 10:43 am

Posted in

Research, Reproducibility, Data

with one comment

Last week, Thomas Herndon, an economics grad student, published a paper that refuted a renown economics paper authored by two Harvard professors on three accounts:

  1. some data was excluded from the analysis without stating the reasons;
  2. during processing of the data, a debatable method for weighting the data was used;
  3. there was a “coding error” – The authors had used MS Excel for their analysis and used a seemingly wrong range of cells for one calculation;

As far as I can tell, Mike Konczal was the first to write about the freshly published paper on April 16th.

First, Reinhart and Rogoff selectively exclude years of high debt and average growth. Second, they use a debatable method to weight the countries. Third, there also appears to be a coding error that excludes high-debt and average-growth countries. All three bias in favor of their result, and without them you don’t get their controversial result.

On April 17th, Arindrajit Dube, assistant professor at economics at the University of Massachusetts, Amherst (the same school of Thomas Herndon). He presents a short and concise analysis of the reasoning behind Herndon’s paper. One key analysis of his relates to the fact that different ranges of the data have a varying degree of dependence. In this case, the strength of the relationship [between growth and debt-to-GDP] is actually much stronger at low ratios of debt-to-GDP. From there he goes on to wonder about the causes of this changing relationship.

Here is a simple question: does a high debt-to-GDP ratio better predict future growth rates, or past ones? If the former is true, it would be consistent with the argument that higher debt levels cause growth to fall. On the other hand, if higher debt “predicts” past growth, that is a signature of reverse causality.

NewImage

Future and Past Growth Rates and Current Debt-to-GDP Ratio. Figure’s source.

Looking at the data is one thing, but looking at causal relationships should always be related. A lot of people suggest that making data and analysis methods publicly available would prevent such errors. I agree to some extent. It is nice to see a re-analysis performed in python online. However, why did the authors not see these causal relationships? Did they not have enough time for a rigorous analysis? And would a rigorous analysis not be necessary for research that forms the basis for (current) political decisions?

Josef Perktold frames it in slightly different words (and also links to a post by Fernando Perez that examines the role of ipython and literate programming on reproducibility):

[…] it’s just the usual (mis)use of economics research results. Politicians like the numbers that give them ammunition for their position

and

“Believable” research: If your results sound too good or too interesting to be true, maybe they are not, and you better check your calculations. Although mistakes are not uncommon, the business as usual part is that the results are often very sensitive to assumptions, and it takes time to figure out what results are robust. I have seen enough economic debates where there never was a clear answer that convinced more than half of all economists. A long time ago, when the Asian Tigers where still tigers, one question was: Did they grow because of or in spite of government intervention?

Stephen Colbert, of course, has his own thoughts, and has invited Thomas Herndon to chat with him:

Written by Claus

April 25th, 2013 at 8:50 am

Posted in

Pythonanywhere Update

with 2 comments

Pretty much one year ago, I had written about Pythonanywhere. I thought I’d try their servers out again. Here is what I noticed, after not having used it for a year:

  • the connection to the dropbox folder still works
  • it is faster (see updated chart)
  • they updated python to 2.7.3 (and 3.2 if you like), and numpy to 1.6.2
  • it is possible to run ipython and ipython notebooks!
  • it is possible to schedule tasks!
  • they extended possibilities for web servers and mysql
  • latex, git integration

Awesomeness! 😉

comparison pythonanwhere vs. local machine

Written by Claus

January 3rd, 2013 at 11:09 am

Posted in