Some useful Xpath expressions to extract correct element from Html/XML.
---------------------------------------------------------------------------------

I faced many issues while trying to learn to write Xpath expressions when i was trying to parse correct elements from Html and XML documents.

Here are some of the long and complex XPaths with their explanations.

1. xpathApply(doc,"//p[not(ancestor::footer[@class='footer footer-intl'])][not(@class='email-signup-description')]",xmlValue)

Explanation: Get the value of those "p" nodes, which do not have "footer" ancestors with class aatributes value as "footer footer-intl" and which do not have "footer" ancestors with value of class attribute as "email-signup-description".

2. xpathSApply(doc,"//p[not(contains(@class,'promo__summary'))]", xmlValue)

Explanation: Get the value of those "p" nodes, where value of "class" attribute does not contain the text "promo__summary".

3.xpathApply(doc,"//p[not(ancestor::div[@class='story'])][not(ancestor::div[@class='disclaimer'])][not(ancestor::div[@class='footer-policy-links'])]",xmlValue)

Explanation: Get the value of those "p" nodes where following conditions are met

  •   "div" element which has "class" attribute as 'story' should not be an ancestor.
  •   "div" element which has "class" attribute as 'disclaimer' should not be an ancestor.
  •   "div" element which has "class" attribute as 'footer-policy-links' should not be an ancestor.

These XPaths will give you an indication on how to parse appropriate node values from Html/Xml.

Comments

Post a Comment

Popular posts from this blog

 I found a very strange anamoly of a code working on Jupyter notebook, but its not working on the app.python from a flask applicatoin. When the type of column names if int instead of string, the str.replace() code works on the jupyter notebook, but does not work in the flask python. Scenario: I have a dataframe having year-months(For example 202001, 202002 etc.) as the column name. Now I want to have a replace code, like, col = [ '201911', '201912','202001', '202002', '202003', '202004', '202005', '202006', '202007', '202008','202009','202010']  the below code works on the jupyter notebook, but fails on the flask. data_df[col] = data_df[col].replace(to_replace=['-',' ','\*'], value='', regex=True).astype(float)
Read more
PGDBA at IIM Calcutta, 1st semester at ISI Calcutta. ------------------------------------- We reached ISI campus on 19th July. We were given accommodation at 4th and 5th floor of CD Deshmukh Bhavan. It was a twin sharing accommodation. Every room was well furnished and spacious with attached bathrooms. Round the clock Wifi and  Lift facility were available for us. Mess facility was arranged at Rani bhavan which was 3-4 minutes walk from our hostel. Our first semester classes started on 20th July 2015.Our class schedule was not very hectic. We had two 2-hours classes per day. But as time passed we started having extra classes in the evening and it started getting hectic. Initially our classes started at NAB-1 at ground floor of AN Kolmogorov bhavan. Buildings at ISI are named after personalities. Size of batches are very small at ISI, Normally they have close to 20 students in a batch(It shows hot competitive it is to get into ISI :-)). So classes are of small size. Ours was l...
Read more
Facebook extraction using R. ( For beginners of Facebook App development).  ---------------------------------------------------------------------------------------------- It took me lot of time to authenticate with Facebook when i was trying to extract data from facebook using Rfacebook package in R language. An important step which is not documented properly while authenticating our APP using R code. After creating the Application, Go to Settings and click on "add platform". Their you will find a textbox with name "Site URL"  fbOAuth(app_id = "XXXXXXXXXXXXXX",app_secret = "XXXXXXXXXXXXXXXXX") When we run above code, on console with get a small link. "http://localhost:1410/ ", This link has to be pasted in  Site URL text box in App settings and saved. After we do this, we should press "Enter" on the R console, which lets this API authenticate the APP on Facebook. Once this is done the APP is authenticate and a browse...
Read more