J2Ee Hosting, Sun Java Developing, Domain Name Hosting Blog

928Part VCase StudiesHarvesting dataThe code in the preceding section assumes that you want to transfer data from one databaseto another but what if, instead, you need to dump data from a bunch of static HTML or textfiles into a database? In that case, you have to write yourself a little minispider. Before you try this, you should know that harvesting data from existing documents is aninherently difficult task. If you ve been maintaining 500 static HTML pages for any length oftime, there s very likely to be some irregularities in the files that make it more difficult to pickout the pieces that you want. Sometimes even a simple typo or case change causes problems. Harvesting data this way could still save you some time, however, in certain circumstances. Basically what you want to do is parse text by looking for unique patterns. You hope theinformation you want to harvest is identifiable by such a pattern. The process is very similarto navigating by landmarks, and only human labor can parse the structure of your documentswell enough to pick out the patterns. After a person has clearly picked out a pattern, how- ever, PHP can help you with the repetitive grunt-labor of applying this patterned searching toa whole bunch of documents. Say, for example, that you have several hundred individual HTML press release pages thatyou now want to turn into a database. You analyze the pages and discover that the headlineis always located inside

tags and is the only thing on each page marked by such tags. Inthis case, it would be very easy to pick out the headline from each page. Unfortunately, this isa deliberately easy example in many cases, the actual pattern you need to find is morelikely to be something like look for this string; then it is two paragraphs after that, betweenthe third and fourth linebreaks. Here is a sample snippet of HTML from the old MysteryGuide.com:

REVIEW

Rating: 4 (Very good)

C hesterton s Man53 Please visit Domain Name Hosting services for high quality webhost to host and run your jsp applications. Posted in Domain | No Comments » Free web design - 927Chapter 47Converting Static HTML Sites// First see if Monday, April 21st, 2008 927Chapter 47Converting Static HTML Sites// First see if this subgenre is already in the database$query = SELECT Sub_id FROM subgenreWHERE Sub_name = $Subgenre[$i] ; $result = mysql_query($query); if (mysql_num_rows($result) == 0) { // If not already there, add the subgenre$query = INSERT INTO subgenre VALUES( NULL, $Subgenre[$i] ) ; $result = mysql_query($query); if (mysql_affected_rows() != 1) { echo Problem inserting subgenre $Subgenre[$i] ; } else { $subgenre_id = mysql_insert_id(); } } else { $subgenre_id = mysql_result($result, 0, 0); } // Now associate the subgenre with the book$query = INSERT INTO book_subgenre VALUES( NULL, $book_id, $subgenre_id) ; $result = mysql_query($query); if (mysql_affected_rows() != 1) { echo Problem inserting book_subgenre data for $B_titleand $Subgenre[$i] ; } } } } ?> Notice that we deal with several problematic but common situations in this script, such as: .Changing strings (M/F) into integers (1/0) for more compact storage. .Turning one field into two fields (Author_nameinto A_firstnameand A_lastname). .Handling duplicate information (same author for multiple books). .Turning three different fields (Subgenre1, Subgenre2, Subgenre3) into three rows inthe same table. .Making a nonrelational database truly relational (book_authorand book_subgenretables). Please visit our professional web hosting services to find out about cheap and reliable webhost service that will surely answer all your demands. Posted in Domain | No Comments » 926Part VCase StudiesListing 47-2(continued) $author_id = mysql_insert_id(); //Need (Best web design) Monday, April 21st, 2008 926Part VCase StudiesListing 47-2(continued) $author_id = mysql_insert_id(); //Need this for book_author} } else { $author_id = mysql_result($result, 0, 0); } // Book data// We can assume each book is unique$query = INSERT INTO book VALUES( NULL, $B_title , $B_year , $B_rating, $B_action, $B_humor, $B_romance, $B_sex, $B_violence, $B_reviewer , $B_pages, $B_reviewdate , $B_ISBN , $B_movie , $B_awards , $B_review , $B_blurb ) ; $result = mysql_query($query); $book_id = mysql_insert_id(); //Need this for book_authorif (!$book_id || $book_id == ) { echo Problem inserting book data for $B_title ; } // Associate book and author$query = INSERT INTO book_author VALUES( NULL, $book_id, $author_id) ; $result = mysql_query($query); if (mysql_affected_rows() != 1) { echo Problem inserting book_author data for $B_title ; } // Subgenresfor ($i = 1; $i <= 3; $i++) { if ($Subgenre[$i] == ) { continue; } else { You need excellent and relaible webhost company to host your web applications? Then pay a visit to Inexpensive Web Hosting services. Posted in Domain | No Comments » 925Chapter 47Converting Static HTML (Web space) Sites$B_rating = $l_arr[10]; $B_action Sunday, April 20th, 2008 925Chapter 47Converting Static HTML Sites$B_rating = $l_arr[10]; $B_action = $l_arr[17]; $B_humor = $l_arr[18]; $B_romance = $l_arr[19]; $B_sex = $l_arr[20]; $B_violence = $l_arr[21]; $B_reviewer = addslashes($l_arr[23]); $B_pages = $l_arr[24]; $B_reviewdate = $l_arr[25]; // Reformat the review date from m/d/yy to yyyy-mm-dd$date_arr = explode( / , $B_reviewdate); $B_reviewdate = date( Y-m-d , mktime(12, 30, 0, $date_arr[0], $date_arr[1], $date_arr[2])); $B_ISBN = $l_arr[26]; $B_movie = addslashes($l_arr[27]); $B_awards = addslashes($l_arr[28]); $B_review = addslashes($l_arr[29]); $B_blurb = addslashes($l_arr[30]); // Subgenre info$Subgenre[1] = addslashes($l_arr[5]); $Subgenre[2] = addslashes($l_arr[6]); $Subgenre[3] = addslashes($l_arr[7]); // Enter data into database// ———————— // Author data// First see if this author is already in the database$query = SELECT A_id FROM authorWHERE A_firstname = $A_firstname AND A_lastname = $A_lastname ; $result = mysql_query($query); if (mysql_num_rows($result) == 0) { // If not already there, add the author$query = INSERT INTO author VALUES( NULL, $A_firstname , $A_lastname , $A_gender, $A_nationality , $A_interview ) ; $result = mysql_query($query); if (mysql_affected_rows() != 1) { echo Problem inserting author data for $A_firstname$A_lastname ; } else { Continued53 If you are looking for affordable and reliable webhost to host and run your business application visit our ftp web hosting services. Posted in Domain | No Comments » Next Entries » You are currently browsing the J2Ee Hosting, Sun Java Developing, Domain Name Hosting Blog weblog archives for April, 2008. Archives May 2008 April 2008 March 2008 February 2008 January 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 May 2007 April 2007 Categories Domain (513) J2Ee Hosting, Sun Java Developing, Domain Name Hosting Blog is proudly powered by cheap hosting Entries (RSS) and Comments (RSS).