Discussion Forums  >  WebViews and HTML for Mobile

Replies: 8    Views: 114

Kryten
Lost but trying
Profile
Posts: 8
Reg: Jun 09, 2012
Birmingham
7,430
06/12/12 05:28 AM (13 years ago)

Data scraping

Hi guys, I'm new here and am just learning Java and Eclipse. I don't know if this is really the best catagory for this question or even if the questionis really appropriate for these forums at all, but here goes.... Basically, I was wondering if anyone could point me in the right direction here. How do apps such as any of the football apps or perhaps the weather apps even, go about gathering their information for live updates. Is it via a data scrape of their own sites, other commercial sites or what? If it is via a data scrape, is this something you would program in Jave/Eclipse (for Android here of course) or by some other method. Sorry if the question is a bit vague but I hope you all get the gist of what I'm taking about. Cheers
 
Aaronus Maximus
Lost but trying
Profile
Posts: 62
Reg: May 01, 2012
Tunbridge Wells
7,870
like
06/12/12 05:51 AM (13 years ago)
Afternoon Kryten Funny you should mention that as I am currently creating an application that im hoping will provide live updates for football teams. I'm sure the android monkeys on here will come up with something spectacular (looking at you Fred) but from what I have looked into it seems the best way is to either 1) use a custom url page of a normal website that provides updates (footballwebpages/bbc/skysports etc) or 2) use an html script from one of those websites to automatically load the data you want into an html 5 screen. This is a method I am currently working on to provide me with live league "as it stands" tables on matchdays. Hope this helps! Cheers, Aaron
 
mysps
Code is Art
Profile
Posts: 2082
Reg: May 14, 2011
Palma
33,320
like
06/12/12 06:26 AM (13 years ago)
You would have to use API's to work with Buzz... As for sports for example: http://developer.espn.com/ weather: http://www.wunderground.com/weather/api/ Do a google search and you'll find tons of api's no use of scraping :)
 
Kryten
Lost but trying
Profile
Posts: 8
Reg: Jun 09, 2012
Birmingham
7,430
like
06/12/12 06:30 AM (13 years ago)
Aha, thanks guys. I had a feeling API's were the way forward, I was just hoping for a relatively simple cost free solution. Oh well, you get nothing for nothing nowadays!
 
Paul Rogers
Android Fan
Profile
Posts: 2524
Reg: May 30, 2011
UK
35,740
like
06/12/12 06:58 AM (13 years ago)
It's totally possible to use regex in your project in eclipse. Stackoverflow.com should provide some insights.
 
Aaronus Maximus
Lost but trying
Profile
Posts: 62
Reg: May 01, 2012
Tunbridge Wells
7,870
like
06/12/12 06:58 AM (13 years ago)
One thing I would suggest kryten, Open up a completely free acocunt with footballwebpages, they have the functionality to publish content from any page on their site, to your own site (or in this case an app). I have been going down this path and its pretty much fully functional, havent paid a penny yet :) Cheers, Aaron
 
Fred@mySkylla com
Android Fan
Profile
Posts: 5259
Reg: Oct 03, 2011
location unknow...
62,560
like
06/12/12 07:00 AM (13 years ago)
@Aaronus, thanks for the shout out, but as you can see, others have more and better ideas than I. Primarily what I do is make document your ideas into my guide. Many excellent ideas posted on the forum but without knowing the correct search terms the post scrolls away and usually never is seen again. I can up with my "floating buttons" solution only to stumble across the similar (and better) solution that had been posted by David @Buzztouch about a year earlier. He used menu (list), I used menu (button). Great espn and wunderground API find. Nice entries for my guide. Fred
 
Kryten
Lost but trying
Profile
Posts: 8
Reg: Jun 09, 2012
Birmingham
7,430
like
06/12/12 07:05 AM (13 years ago)
Aha, thanks guys. I had a feeling API's were the way forward, I was just hoping for a relatively simple cost free solution. Oh well, you get nothing for nothing nowadays!
 
Kryten
Lost but trying
Profile
Posts: 8
Reg: Jun 09, 2012
Birmingham
7,430
like
06/12/12 07:58 AM (13 years ago)
Ok, thanks again guys. Some useful info there. Aaron, I was only using football as an example. I'm not necessarily going to go that route at the moment, just exploring possibilities. Footballwebpages does look like a good site though so thanks. Raveyd, Stackoverflow looks like a great resource so again, thanks - Apple fan with a melon on your head, love it. I'm sure it can be done with data scraping from what's been said but I have used data scraping in the past and it can be messy from experience. I have never used API's before but I'm fairly convinced that it's likely to be the most straightforward and consequently most efficient method. So, all I have to do now is to learn how to do it - piece of cake!
 

Login + Screen Name Required to Post

pointerLogin to participate so you can start earning points. Once you're logged in (and have a screen name entered in your profile), you can subscribe to topics, follow users, and start learning how to make apps like the pros.