planetwater

The mac has recently had its 30th birthday. I am too young to remember its first birthday, but I am old enough to feel the pain! I’ve never used anything pre OSX. One piece of software (as apps have previously been called) has been with me since the beginnings: omnioutliner. I won’t get into fights of outlining versus mindmapping. I think both have their place, and I also think that mindnode is an awesome software.

Taking notes that contain equations has been and is my bread and butter. When laptops were finally portable enough and affordable enough to be taken into a classroom, this was a joy for me, because I could use then omnioutliner for my notes — back then, a little app called “oo3eq” facilitated the combination of equations and omnioutliner. This is how it worked:

• type your equation in latex into the note field of omnioutliner
• hit a button, and a pdf would appear in the row that belongs to the note
• the equation would still be in the note, which I found very useful for future reference.
• having a pdf of the equation visible directly above the note, was much more pleasing to the eye, and helped following the notes that I took tremendously.

I did try to reproduce this behaviour, and you can check it out on github. There is an applescript that gets the content of the selected note as a string, which must be a valid latex math expression. This string is passed to a python script that generates a valid latex file, which is processed in the script. The result is a pdf file. Back in applescript, this pdf file is pasted as an attachment into the selected row.

Here is a screenshot how this can look like:

I am running this script via a key trigger in quicksilver, as described here. This works well, but is not nearly as nice as the beautiful icons that came with “kinkless gtd“, which later turned into omnifocus.

This is my reasoning for doing it:

• omnioutliner 4 has been released somewhat co-incidental with the recent mac birthday
• oo4 continues to have strong applescript support
• oo3eq is not around anymore; at least I couldn’t find it, and the version I still had around did not work anymore on my current setup
• the whole thing is reasonably quick, and I learned a few things about applescript and python

This post has to do neither with water nor with statistics. But because it’s been christmas break, I found some time to do some fun stuff with python:

I wrote a python script that downloads the issue of my choice of a pdf on my iPad. Partcularly, the script

• lets me chose which issue of the paper (relative to the current day) I want to download
• checks if that file might already exist on a relevant dropbox folder – checks if that file exists on the newspaper’s server (there is no paper on a holiday, f.ex) – if checks are ok, then the pdf is downloaded locally – saves the pdf in a selected folder within my dropbox – opens Goodreader, downloads the current file from dropbox (the proper url of the file needed to be determined) – deletes the downloaded pdf locally within pythonista

The relevant modules are the dropbox module of pythonista and the requests module to download the pdf with authentication.

The rationale behind attempting this was two fold:

• using python on my iPad: I have downloaded pythonista a little while ago, but haven’t had time to play with it

• since it has been possible, I have been subscribing to a German daily newspaper, which can be download as a pdf. I guess they are trying to foster the use of iPaper. I tried that for the last couple of weeks using iBooks on the iPad, and I don’t like it for two reasons:

  1. the iPaper files are larger than the pdfs, hence they take noticeably longer to load.
  2. I found it to be almost impossible to get the iPaper files out of iBooks on the iPad (to some storage place, for example).

The tipping bucket that led me to actually do this was that since a little while it is not possible anymore to download the pdfs directly from within Goodreader

Goodreader came out with an updated version two days ago. This makes the script even more powerful. This update solved issues related to the rendering of pdfs. I am not getting money from Goodreader, but I consider it to be the most useful software I have on my iPad. It particularly integrates awesomely with directories on an OSX Server.

Downloading the pdf

This is surprisingly easy using the requests package, that I just found out that it existed. Basically, downloading a pdf from a known url comes down to one line:

r = requests.get(url, auth=('XXX', 'YYY'))

Saving to Dropbox

This is step is a little bit more involved:

• Before you start you have to register an ‘app’ in the developer part of dropbox. This will get you an APP_KEY and an APP_SECRET. In the python API you also need a token, which is from what I understand a string consisting of the name of your app in dropbox concatenated with '.token'.
• When you’ve got that, I found some code on github that deals with the authentication process.

• Finally, within the python module within pythonista, saving a file comes down to another one liner basically (with some preparations):

  sess = dropbox.session.DropboxSession(APP_KEY, APP_SECRET, ACCESS_TYPE) 
  client = dropbox.client.DropboxClient(sess) 
  f = open(filename) 
  response = client.put_file('/Apps/Pythonista/' + filename, f)
  

Open Issuess

• the link between pythonista and good reader via dropbox works reasonably well. Would be nice to have a direct link (along the lines of ‘open in’).
• it’s possible to create little home scree icons that call your code, also with command line arguments. As far as I can tell, you have to fix a certain value of the command line argument to one icon… that seems a bit against the flexibility of command line arguments
• the time delta doesn’t treat boundaries of months correctly fixed by the first commit
• not sure if raise Exception is really the way to go if a) the file exists already on dropbox or b) if there doesn’t exist a paper on the desired date

Conclusions

• Pythonista offers a tool to achieve things that otherwise would be next to impossible on the iPad – at the same time, one has to wonder, why such things are forced to be impossible to begin with – writing code on the iPad without a dedicated external keyboard is… challenging. Despite the good editor of Pythonista.

The script is available on github.

Welcome 2014!

I’ll start this year by pointing to two articles from the New York Times that point out the importance of uncertainty:

1. Monte Carlo to try and Find Fisherman who Fell off his Boat

This story in the New York Times Magazine describes the rescue for a fisherman who fell off his boat into the Atlantic while his colleagues were sleeping. The human aspects play the most important part, but one section describes how the US Coast Guard uses a Monte Carlo based simulation tool to predict the most likely locations where to search for the fisherman floating in the ocean. I found this story via John Gruber.

2. Inconvenient Uncertainties

In this comment, on another article published in the New York Times, Gernot Wagner tries to make the point that science in general, particularly environmental science is inherently uncertain . The title of the original article that he refers to is “By 2047, Coldest Years May Be Warmer Than Hottest in Past, Scientists Say.” Wagner tries to make the point, that science in general and climate science in particular, are uncertain. Will this threshold be passed exactly in 2047 or might it be around 2050? He continues to state that

The scientific method imposes some order, but in the case of climate change, that order is probabilistic. For the sake of science and the planet, we should not become distracted by a false sense of certitude. Imprecise truths are the most inconvenient ones. We know enough to act now. What we don’t know should prompt us to even more decisive action.

• RT @modernscientist: Woah! RT @PhilippBayer: Mathematica will be bundled with Raspbian, for free! http://t.co/1OYZDhZX4H 2013-11-22

• ample room for improvement, but cool concept: Monte Carlo with #Lego in laundry machine: http://t.co/WlGWKp36B0 2013-11-20

• RT @Riotbuddha: Diese Windräder verschandeln uns die ganze Landschaft. (oben rechts) http://t.co/0im6WSkr15 2013-11-08
• RT @EricHolthaus: Super Typhoon #YolandaPH is officially off the charts. http://t.co/XGWeFv3GAJ 2013-11-08
• #climate #history stored in #boreholes – saw talk recently; get ready for #COP19 http://t.co/L9oeJHOT4g http://t.co/zI6AM0zUfJ 2013-11-10
• finally: some #stats on hard drive disk failures: http://t.co/6GlQedEXqY 2013-11-14

The UN framework convention on climate change will start tomorrow in Warsaw, Poland. By chance, but fitting to the occasion, I heard a talk on friday given by David Chapman from the University of Utah.

He opened his talk with a remarkable chart, shown below:

This chart shows the thermal gradient measured in multiple boreholes (labelled ETK, FCK, TUL, etc.) that were drilled in northern Alaska. At depth, ranging from about 125m to 175m bgs., temperature increases linearly with increasing depth. However, for shallower depths between about 70m to 100m bgs, there is a deviation from the linear slope recognizable, indicated by the grey shaded areas. The authors found that the deviation from the expected temperature gradient is a few degrees warmer (between 2.2C and 7.4C) in the last 100 years or so.

What a great “reservoir” the earth is! The chart above is from this paper:

Lachenbruch, Arthur H, and B Vaughn Marshall. 1986. “Changing Climate: Geothermal Evidence From Permafrost in the Alaskan Arctic.” Science 234 (4777) (November 7): 689–696. doi:10.1126/science.234.4777.689.

Note that this was published about one year before Hansen’s and Lebedeff’s land-mark paper. Based on Lachenbruch’s and Marshall’s work work, Chapman continued to do related work, particularly with meteorological measurements, in the west of Utah.

• RT @hukl: Binary trees – they really exist!! http://t.co/aQOIpDj8ne (via @IAmMarth) 2013-11-06
• RT @Earth_Pics: Sand under a microscope with 250x magnification. http://t.co/7tXhWxjwLD 2013-11-06
• RT @lintool: When I see this, I think: what an optimization nightmare… too many local minima http://t.co/dlP12KkRun 2013-11-07

• Via @zenrainman @WaterWired Srikakulam, India: new rainfall record,receives 1263 mm in 6 days. http://t.co/rfPFg2xPef @TheHindu 2013-10-29

• this is awesome! RT @danielhfrank: @wesmckinn use heapq.nlargest in a pinch? Agree that a NumPy version would be good to have though 2013-10-21
• RT @EricHolthaus @NOAA #extreme precip in Bolder, CO: rare event on multiple time scales http://t.co/3sYL5dACUi (1/2) http://t.co/P7p7yvs0Jk 2013-10-24