DataIO
A few weeks ago my employer helped the NY State Senate parse the MTA budget information into a machine searchable format. (The MTA originally published the budget as a PDF.) To parse the PDF I used a utility called pdftohtml to first convert the PDL into an XML document. I then used the python library lxml to convert the document into a set of csv files. The results of this labor can be seen on TOPP’s data site.
Soon after I published this data, however, I was told by a number of people that the data would be more useful if presented in another format. At first I just started creating a bunch of command line python scripts that would suck in these csv files, and spit them out in different formats. I quickly realized that I could accumulate these scripts and create a quick and dirty web application.
Over a few train rides I created an application called DataIO, and today, I finally got a chance to upload it to Google App Engine. The application is pretty simple to interact with; instructions are located on its front page.
Currently the application can only transpose data and multiply the data set by a given factor. I hope to soon add a jsonp api that will make it trivial to convert a given data set into a format that plays nice with google charts and flot.
The code for this application is hosted at bitbucket.
Just for fun, here is some data from data.topplabs.org, sent through dataIO.
Operating Revenue (transposed and multiplied by 1000000):
http://www.dataio.org/data/f?transpose=true&multiplication_factor=1000000&multiplication_start_row=1
Total Receipts by Agency (transposed and returned in json):
http://www.dataio.org/data/Bt?format=json&transpose=true
Bridges and Tunnels Summary of Total Budgeted Debt Service (multiplied by 100 and returned in csv):
http://www.dataio.org/data/IM?multiplication_factor=100&multiplication_start_row=1&format=csv
Hi Anil,
We are talking about similar issues at http://citability.pbworks.com/PDF I am going to crosslink your post there. I am also working on getting my PDF expert that parsed all the PDF’s from whitehouse.gov and change.gov to participate.
@Silona: Neat. I actually have done some more work on DataIO. The additional features I added are outlined in this post: http://topplabs.org/civichacker/2009/05/dataio/
We are glad to offer you service of dumps selling.
PRICE DUMPS TRACK 1&2:
USA:
VISA AND MASTERCARD CLASSIC FOR:
1-25 – 15$ each
25-50 – 10$ each
50-100 – 7$ each
VISA AND MASTERCARD GOLD/PREMIUM/BUSINESS/CORPORATE/SIGNATURE ETC. FOR:
1-25 – 30$ each
25-50 – 20$ each
50-100 – 15$ each
AMEX ALL TYPES:
1-50=30$ each
50-100=20$ each
100-500=15$ each and less
All discovers cost 25$ each.
————————————–
CANADA:
VISA AND MASTERCARD CLASSIC FOR:
1-25 – 20$ each
25-50 – 15$ each
50-100 – 10$ each
VISA AND MASTERCARD GOLD/PREMIUM/BUSINESS/CORPORATE/SIGNATURE ETC. FOR:
1-25 – 30$ each
25-50 – 20$ each
50-100 – 15$ each
—————————————
EUROPE:
VISA AND MASTERCARD CLASSIC FOR:
1-25 – 60$ each
25-50 – 50$ each
50-100 – 45$ each
VISA AND MASTERCARD GOLD/PREMIUM/BUSINESS/CORPORATE/SIGNATURE ETC. FOR:
1-25 – 90$ each
25-50 – 80$ each
50-100 – 75$ each
—————————————-
UK:
VISA AND MASTERCARD CLASSIC FOR:
1-25 – 55$ each
25-50 – 50$ each
50-100 – 40$ each
VISA AND MASTERCARD GOLD/PREMIUM/BUSINESS/CORPORATE/SIGNATURE ETC. FOR:
1-25 – 80$ each
25-50 – 70$ each
50-100 – 65$ each
—————————————–
ASIA:
VISA AND MASTERCARD CLASSIC FOR:
1-25 – 50$ each
25-50 – 45$ each
50-100 – 40$ each
VISA AND MASTERCARD GOLD/PREMIUM/BUSINESS/CORPORATE/SIGNATURE ETC. FOR:
1-25 – 70$ each
25-50 – 60$ each
50-100 – 55$ each
————————————————-
100 AND MORE, INDIVIDUAL PRICE FOR EVERY COSTUMER.
Rules:
-Sending dumps during 30 minutes after full payment received.
-Please do not ask for anything else than dumps because I don’t sell something I don’t have.
-I only deal with serious people and business man so Rippers do not waste your time for asking FREE SAMPLES.
-People who want to see the quality of my dumps, just ask me and I’ll sell some pcs to test (Only with Webmoney or Libertyreserve).
-I accept Webmoney, Libertyreserve (no min order) and Westernunion, Moneygram.
-Dumps are skimmed and they are fresh skimmed.
-All dumps can be checked and given to you 00 Approved only
-I have my own rights to refuse to offer my service and deal with a non serious buyer.
-I replace bad dumps ( PICK UP , STOLEN CARDS….).
-I replace only within 48h after your purchase.
contacts:
my ICQ: 551628393
my e-mail: trackdumps@gmail.com
phone number: only for constant buyers