Comments on: Create your First Web Scraper to Extract Data from a Web Page http://blog.outwit.com/?p=55 Harvest the Web Wed, 25 Aug 2010 16:22:12 +0000 hourly 1 http://wordpress.org/?v=3.0.1 By: jcc http://blog.outwit.com/?p=55&cpage=1#comment-1964 jcc Mon, 17 May 2010 07:58:13 +0000 http://blog.outwit.com/?p=55#comment-1964 @Jim: The Apply to URL in the scraper can be a partial URL, or a regExp or just a string to be found in the current URL for the program to decide if the scraper applies to the page. The idea is that once on a page, simply going to the 'scraped' view will execute the process. Imagine you put 'mySite.com/search?' in the apply to URL, then the scraper will apply to any result pages on mySite ; in this case, we cannot load the page as we only have a partial URL. Macros, however will allow this in the Pro version. @Jim: The Apply to URL in the scraper can be a partial URL, or a regExp or just a string to be found in the current URL for the program to decide if the scraper applies to the page. The idea is that once on a page, simply going to the ‘scraped’ view will execute the process. Imagine you put ‘mySite.com/search?’ in the apply to URL, then the scraper will apply to any result pages on mySite ; in this case, we cannot load the page as we only have a partial URL. Macros, however will allow this in the Pro version.

]]> By: jcc http://blog.outwit.com/?p=55&cpage=1#comment-1963 jcc Mon, 17 May 2010 07:46:43 +0000 http://blog.outwit.com/?p=55#comment-1963 For this, you can use the right arrow (next) or the double right arrow (browse) if they are active. You can also collect a list of URLs in your catch for instance, select them and use the right click menu option 'Browse through selected URLs'. (NOTE: For technical questions, please, do use the feedback link rather than posting a comment to the blog for support tickets to be followed.) For this, you can use the right arrow (next) or the double right arrow (browse) if they are active. You can also collect a list of URLs in your catch for instance, select them and use the right click menu option ‘Browse through selected URLs’.

(NOTE: For technical questions, please, do use the feedback link rather than posting a comment to the blog for support tickets to be followed.)

]]> By: Jim http://blog.outwit.com/?p=55&cpage=1#comment-1960 Jim Sun, 16 May 2010 04:04:05 +0000 http://blog.outwit.com/?p=55#comment-1960 One other observation: since the scraper saves the target URL, it seems it might be a good idea to have it go to that URL automatically when executed rather than having to copy and paste it into the address field .. ? One other observation:

since the scraper saves the target URL, it seems it might be a good idea to have it go to that URL automatically when executed rather than having to copy and paste it into the address field .. ?

]]> By: Jim http://blog.outwit.com/?p=55&cpage=1#comment-1959 Jim Sun, 16 May 2010 03:57:47 +0000 http://blog.outwit.com/?p=55#comment-1959 LOVE this app! Thanks! Is there a way to catch all of the return from a paged view? For example, I am working with a search which returns 50 results per page .. but 250 pages. I'm not quite sure how to have the scraper iterate thru all of the pages. Can you help? LOVE this app! Thanks!

Is there a way to catch all of the return from a paged view?
For example, I am working with a search which returns 50 results per page .. but 250 pages.

I’m not quite sure how to have the scraper iterate thru all of the pages.

Can you help?

]]> By: pr http://blog.outwit.com/?p=55&cpage=1#comment-1948 pr Thu, 22 Apr 2010 06:32:48 +0000 http://blog.outwit.com/?p=55#comment-1948 Hi Yigal, Since the last version, it's possible to keep HTML tags in scrapers (also in tables, lists). You'll see a "Clean data" checkbox in the bottom panel, just uncheck it. We'll probably work later on getting text in the email view. Cheers, Hi Yigal,

Since the last version, it’s possible to keep HTML tags in scrapers (also in tables, lists). You’ll see a “Clean data” checkbox in the bottom panel, just uncheck it.
We’ll probably work later on getting text in the email view.

Cheers,

]]> By: jcc http://blog.outwit.com/?p=55&cpage=1#comment-1874 jcc Fri, 26 Mar 2010 09:33:33 +0000 http://blog.outwit.com/?p=55#comment-1874 Hello and thanks. (Please, do use our feedback form in outwit.com for this type or message, it allows us to keep our bug/wish list up to date.) We are replacing returns by ";" (after having replaced all ";" by "," ), to make cut and paste to a spreadsheet easier. It is not a completely satisfying solution but it is a better choice in most cases than simply removing the returns, as it allows you to recover a little of the original layout in the destination program, replacing back ";" by returns. We are adding a "clean text" option (checked by default) in all views of the next version. It means that, if you uncheck it, you will be able to keep html tags like <br> in the scraped text. Actually, this gave us a pretty simple idea: we will add a find/replace function in all datasheets of a next version. It should help in these cases. Cheers, JC Hello and thanks. (Please, do use our feedback form in outwit.com for this type or message, it allows us to keep our bug/wish list up to date.)
We are replacing returns by “;” (after having replaced all “;” by “,” ), to make cut and paste to a spreadsheet easier. It is not a completely satisfying solution but it is a better choice in most cases than simply removing the returns, as it allows you to recover a little of the original layout in the destination program, replacing back “;” by returns. We are adding a “clean text” option (checked by default) in all views of the next version. It means that, if you uncheck it, you will be able to keep html tags like <br> in the scraped text.

Actually, this gave us a pretty simple idea: we will add a find/replace function in all datasheets of a next version. It should help in these cases.

Cheers,

JC

]]> By: ; in place of Line Breaks http://blog.outwit.com/?p=55&cpage=1#comment-1872 ; in place of Line Breaks Thu, 25 Mar 2010 19:31:48 +0000 http://blog.outwit.com/?p=55#comment-1872 I see ";" in place of all the line breaks when using the scrapers. How do we avoid that or remove the ";"? By the way, great tool! I see “;” in place of all the line breaks when using the scrapers. How do we avoid that or remove the “;”? By the way, great tool!

]]> By: Yigal http://blog.outwit.com/?p=55&cpage=1#comment-1825 Yigal Sun, 21 Feb 2010 00:03:28 +0000 http://blog.outwit.com/?p=55#comment-1825 Hi, Thanks for the amazing program. I do miss one feature though. The emails view is good, but lacks the text that is displayed for those emails (Usually the name of the person for which this email belongs). It could be solved either in the emails view or in the scrapper. But the scrapper lacks the ability to catch the HTML tags, so I can catch the name of the person but not its email. I would be grateful if you could add one of these features. Hi,

Thanks for the amazing program. I do miss one feature though.
The emails view is good, but lacks the text that is displayed for those emails (Usually the name of the person for which this email belongs).

It could be solved either in the emails view or in the scrapper. But the scrapper lacks the ability to catch the HTML tags, so I can catch the name of the person but not its email.

I would be grateful if you could add one of these features.

]]> By: How to Extract Any Web Page Information and Export it to Excel http://blog.outwit.com/?p=55&cpage=1#comment-1180 How to Extract Any Web Page Information and Export it to Excel Sun, 20 Sep 2009 22:53:04 +0000 http://blog.outwit.com/?p=55#comment-1180 [...] is a detailed info on creating your first scraper as well as the post where I found this cool [...] [...] is a detailed info on creating your first scraper as well as the post where I found this cool [...]

]]> By: OutWit Of hub - We Export Any Information From The Web Pages InTo Excel | Seo Sheen http://blog.outwit.com/?p=55&cpage=1#comment-996 OutWit Of hub - We Export Any Information From The Web Pages InTo Excel | Seo Sheen Fri, 14 Aug 2009 06:33:34 +0000 http://blog.outwit.com/?p=55#comment-996 [...] because in it there is a possibility of writing its rules for [parsing] - Create of your Of first Of scraper (instruction with the [...] [...] because in it there is a possibility of writing its rules for [parsing] – Create of your Of first Of scraper (instruction with the [...]

]]>