Download Approach Plates

I've been grabbing them from AOPA, but thanks for the link. Now I just need a script to go grab my usual airports every cycle...
 
AirBaker said:
I've been grabbing them from AOPA, but thanks for the link. Now I just need a script to go grab my usual airports every cycle...
Click to expand...
That's a feature I'd love to have. On AOPA, you an download a group but I'd like more options with different groups as well as automatic print options such as printing two plates on one page and saving paper.
 
Tried it but wasn't successful. Just got a blank page with the browser showing "done"
 
Flightaware also allows downloading all plates for one airport in one PDF bundle.
 
There IS a way to d/l all the charts for a particular airport (I'm not script-literate enough to do it yet) since chartfinder.vatsim.net is doing it.

chartfinder.vatsim.net operates under the disclaimer that it is for simulation purposes only, but if you look at the address when you actually look at their charts, they are pulling from the NACO database as well.

Actually.. A script to d/l and possibly print batches of selected airport charts would be a fun thing for me to learn scripting for my own website. hmmmm..


..... countdown till Jesse has the script completed... 10... 9... 8....
 
CJones said:
..... countdown till Jesse has the script completed... 10... 9... 8....
Click to expand...

Ugh. You just have to challenge me. There goes any chance I had at slacking off this week :) I suspect they got their hands on some database they manipulated into doing what they wanted. I'll look into it some..
 
jesse said:
Ugh. You just have to challenge me. There goes any chance I had at slacking off this week :) I suspect they got their hands on some database they manipulated into doing what they wanted. I'll look into it some..
Click to expand...

I tried stealing their source, but they have their 'go get it' functions hidden to a point that 'I' can't find it. Guess I'll have to think about it from a 'ground-up' standpoint.
 
Write a script that will retrieve and download a selected group of plates that you've filled in and gives you the ability to print them in a pre-chosen format... make it available to PoA members for a contribution to the board. Heck, I'd go for that.

The picky part will be selecting specific plates for a given airport and for multiple airports. Not the entire group for a single airport. For example, I have no need for NDB approaches unless they are GPS overlays. So, I'd like to be able to pick in that fashion.
 
CJones said:
There IS a way to d/l all the charts for a particular airport (I'm not script-literate enough to do it yet) since chartfinder.vatsim.net is doing it.

chartfinder.vatsim.net operates under the disclaimer that it is for simulation purposes only, but if you look at the address when you actually look at their charts, they are pulling from the NACO database as well.

Actually.. A script to d/l and possibly print batches of selected airport charts would be a fun thing for me to learn scripting for my own website. hmmmm..


..... countdown till Jesse has the script completed... 10... 9... 8....
Click to expand...

jesse said:
Ugh. You just have to challenge me. There goes any chance I had at slacking off this week :) I suspect they got their hands on some database they manipulated into doing what they wanted. I'll look into it some..
Click to expand...

CJones said:
I tried stealing their source, but they have their 'go get it' functions hidden to a point that 'I' can't find it. Guess I'll have to think about it from a 'ground-up' standpoint.
Click to expand...

KennyFlys said:
Write a script that will retrieve and download a selected group of plates that you've filled in and gives you the ability to print them in a pre-chosen format... make it available to PoA members for a contribution to the board. Heck, I'd go for that.

The picky part will be selecting specific plates for a given airport and for multiple airports. Not the entire group for a single airport. For example, I have no need for NDB approaches unless they are GPS overlays. So, I'd like to be able to pick in that fashion.
Click to expand...

Are you guys talking about a collaboration or three independent attempts?
 
OK, I've been fooling around with this today and wrote a perl/curl script to download everything and build a mysql database. I've got all the links to the pdfs and still need to download them and write an interface.

It was a bit tricky because the only thing I could figure was to go through state by state and get a list of the charts and follow links for the multiple pages.

Shouldn't be too hard to get the pdfs to start downloading before I go to bed.

There are 16,481 pdfs to download. Just getting the list took 15 min.

So what kind of interface would you like?

I'm thinking of
  • indivdual accounts
  • multiple ways to search including type of approaches (based on parse of name)
  • save a list of selected plates (approaches, minimums, arrrivals, departures)
  • Flag wheter you want all or just the changed, names of deleted, added.
  • Do you want to download or have it emailed to you? Or a list of links could be emailed and you download them from http://www.naco.faa.gov.
This will keep me out of trouble for a while.

Joe
 
Areeda said:
OK, I've been fooling around with this today and wrote a perl/curl script to download everything and build a mysql database. I've got all the links to the pdfs and still need to download them and write an interface.

It was a bit tricky because the only thing I could figure was to go through state by state and get a list of the charts and follow links for the multiple pages.

Shouldn't be too hard to get the pdfs to start downloading before I go to bed.

There are 16,481 pdfs to download. Just getting the list took 15 min.

So what kind of interface would you like?

I'm thinking of
  • indivdual accounts
  • multiple ways to search including type of approaches (based on parse of name)
  • save a list of selected plates (approaches, minimums, arrrivals, departures)
  • Flag wheter you want all or just the changed, names of deleted, added.
  • Do you want to download or have it emailed to you? Or a list of links could be emailed and you download them from http://www.naco.faa.gov.
This will keep me out of trouble for a while.

Joe
Click to expand...

i wanna download. my crappy email server at school chokes on big files and links are long.
 
tonycondon said:
i wanna download. my crappy email server at school chokes on big files and links are long.
Click to expand...
I think that has to be an option a one button "get my update".

Oh if anyone else can't sleep I've attached the list of all the approaches and their links for the March 15 to April 12 set.

Joe

edit: there's a problem with the attachement the corrected one is in post #19.
 
Last edited:
hmm that is actually pretty handy Joe. Not too hard to put into an excel file and then search for the airport you are looking for. i can see conceptually how it would be a pretty easy program to write to pull the files up.
 
Areeda said:
OK, I've been fooling around with this today and wrote a perl/curl script to download everything and build a mysql database. I've got all the links to the pdfs and still need to download them and write an interface.

It was a bit tricky because the only thing I could figure was to go through state by state and get a list of the charts and follow links for the multiple pages.

Shouldn't be too hard to get the pdfs to start downloading before I go to bed.

There are 16,481 pdfs to download. Just getting the list took 15 min.

So what kind of interface would you like?

I'm thinking of
  • indivdual accounts
  • multiple ways to search including type of approaches (based on parse of name)
  • save a list of selected plates (approaches, minimums, arrrivals, departures)
  • Flag wheter you want all or just the changed, names of deleted, added.
  • Do you want to download or have it emailed to you? Or a list of links could be emailed and you download them from http://www.naco.faa.gov.
This will keep me out of trouble for a while.

Joe
Click to expand...


I'm impressed. Have a place to host it yet?
 
AirBaker said:
I'm impressed. Have a place to host it yet?
Click to expand...

I just have a DSL connection with 500K upload so I'll be able to test it and handle fairly light loads. If it becomes popular we will need mirrors but a fully functional system it a way off.

There was a bug in the links download. Acutal count is 17,804. This tab separated file should have them all.

Joe
 

Attachments

  • dttp.zip
    186.8 KB · Views: 25
Areeda said:
I just have a DSL connection with 500K upload so I'll be able to test it and handle fairly light loads. If it becomes popular we will need mirrors but a fully functional system it a way off.

There was a bug in the links download. Acutal count is 17,804. This tab separated file should have them all.

Joe
Click to expand...
Nice work. Insomnia rocks.
First line of data is concatenated to the header line in your .txt file.
 
AuntPeggy said:
First line of data is concatenated to the header line in your .txt file.
Click to expand...
Thank you, it is the log file of the download and I didn't check too closely.

I should point out that the links change every release, perhaps only the directory, in this one it's always 0703.

I've got lots of bugs and new features to work on just to get the database built and maintained.

I'm downloading all the pdfs now. I started it before I went to bed last night but had mysql set to max packet of 1M so it stopped in california.

It seems to be downloading pdfs at about 15 per minute so about 20 hr for the whole thing. Updates will be much easier as each plate is marked Added, Changed, Removed or no change.

It's a fun project. I'll keep you all informed.

Joe
 
Instead of downloading all the pdfs, is it possible to write a crawler that just updates the file information every cycle? Surely, you'll get tired of maintaining by hand eventually.

I never wrote a crawler & wouldn't know how to.
 
AuntPeggy said:
Instead of downloading all the pdfs, is it possible to write a crawler that just updates the file information every cycle? Surely, you'll get tired of maintaining by hand eventually.

I never wrote a crawler & wouldn't know how to.
Click to expand...

Yes that's the idea. Right now all I have are 2 perl/curl scripts that run unattended.

One downloads the description and links to the pdf and the other that downloads the pdfs. After one complete download they will download only those marked added/changed/removed.

Both will be run as a cronjob during the night checking if a new cycle has been released.

My current thinking is that it will be more efficient if the server has all the files but I may change my mind and only keep the ones we need and get others when they're requested.

My current estimate is that the database will be close to 4GB.

Joe
 
Areeda said:
Yes that's the idea. Right now all I have are 2 perl/curl scripts that run unattended.

One downloads the description and links to the pdf and the other that downloads the pdfs. After one complete download they will download only those marked added/changed/removed.

Both will be run as a cronjob during the night checking if a new cycle has been released.

My current thinking is that it will be more efficient if the server has all the files but I may change my mind and only keep the ones we need and get others when they're requested.

My current estimate is that the database will be close to 4GB.

Joe
Click to expand...

Let me know if you want help. I'm more of a PHP guy but have access to plenty of bandwidth and hardware. I was too busy talking to Tristan on the phone last night and completely forgot about this.

Did you more or less just write something that would interpret the HTML information on each page and keep on following?

I can get a dedicated box setup with Redhat ES and plenty of space with bandwidth in pretty short order. I can help with writing some of the GUI type stuff if you get bored or something. But I'd rather use php....
 
Last edited:
jesse said:
Let me know if you want help. I'm more of a PHP guy but have access to plenty of bandwidth and hardware. I was too busy talking to Tristan on the phone last night and completely forgot about this.

Did you more or less just write something that would interpret the HTML information on each page and keep on following?

I can get a dedicated box setup with Redhat ES and plenty of space with bandwidth in pretty short order. I can help with writing some of the GUI type stuff if you get bored or something. But I'd rather use php....
Click to expand...

Thanks Jesse!

I think that will work great. I'm working with Redhat ES 4. The unattended cron intended scripts are perl/curl but the GUI will be PHP (4), my favorite server scripting language also. Underlying DB is mysql 4.1.20.

Right now these scripts are about half done. They only handle the initial downloads. Yes, they interpret (some of ) the HTML and keep following until I get what I want. I'm not sure how robust they will be when the pages start changing, they may need a lot more smarts.

I've never done a completely long distance collaboration so suggestions are ore than welcome. I'm thinking I'll take it to a minimal prototype then CVS it and interested parties can join in. Or I suppose I could send you these and you could work on the UI while I concentrate on maintaining the plate tables.

Joe
 
Wow.. Just mention a 'good' idea and someone here will run with it like a kid with scissors!

I'd also like to stay in the loop of things, if possible. I'm completely illiterate for php right now, but I 'used' to be pretty capable at C++, so hopefully PHP will come fairly easily if I can get a running start at something. I just started back to school (majoring in Mgmt Info Systems) so being literate in PHP would be a good thing to have down the line.

I just set up a new domain/hosting with a fairly good sized bandwidth/storage availability if you need any extra space/testing platform.

Let me know if/when I can help.
 
And, my specialties are software testing and quality control, Java, C++, PHP, mysql, etc.
 
Looks like there are plenty of offers for hosting. I work for a SaaS company that is a host for ISVs.
 
Well i dont know C++ or php or mysql or none of that crap. But I use approach plates so Ill test it out when all you technowizards think you got it figured out :D
 
AuntPeggy said:
And, my specialties are software testing and quality control, Java, C++, PHP, mysql, etc.
Click to expand...

I'm just a user, but back when I used in house stuff, the programmers like me because:

  1. I used ALL of the features,
  2. I invented new ways of using features for other things
  3. I wrote up very detailed descriptions of what I was doing right before the app dumped....
  4. Yes, writing up dumps means I was using big iron apps running on MVS on 360/370
 
AuntPeggy said:
And, my specialties are software testing and quality control, Java, C++, PHP, mysql, etc.
Click to expand...
I test LCD screens' ability to sustain multiple blows by a sledge hammer when the software doesn't work.

I've sent in ideas to Microsoft on IE7 but they were met with a form letter basically telling me my opinion was worthless. I wonder what response I'd get were I in control of a large government agency's software use.
 
lancefisher said:
If only you could download it. The way they've got it now, it's useless for a cockpit resource unless you have airborne internet.
Click to expand...
It's a PDF file for every page. That seems like a very silly move. I can understand it for plates but why not have files for at least a half a state if not an entire state for most? That way you could load that file into a tablet or similar PC for inflight use. Or, even a Palm Treo for that matter. :dunno:
 
Once we get the terminal procedures I want to do the AFD also.

I haven't looked into this yet but does anyone know of Linux utilites that can make a single pdf containing all the pages from multiple pdfs? Is it as easy as 'cat'?

If anyone wants a real challenge how about a utility that can extract the Admnt date from an approach plate?

Joe
 
Bill Jennings said:
I'm just a user, but back when I used in house stuff, the programmers like me because:

  1. I used ALL of the features,
  2. I invented new ways of using features for other things
  3. I wrote up very detailed descriptions of what I was doing right before the app dumped....
  4. Yes, writing up dumps means I was using big iron apps running on MVS on 360/370
Click to expand...

and now z/OS on S/390... I knew I liked you for some reason. :goofy:
 
KennyFlys said:
It's a PDF file for every page. That seems like a very silly move. I can understand it for plates but why not have files for at least a half a state if not an entire state for most? That way you could load that file into a tablet or similar PC for inflight use. Or, even a Palm Treo for that matter. :dunno:
Click to expand...

It's probably a file size issue where the larger files take longer to load.
 
You must log in or register to reply here.
Back
Top