[Gambas-user] How to Disassemble XML/HTML

[Manu]

Le 10/08/2020 ?? 09:48, John Rose a ??crit??:
> What you have there is JSON embedded in a <script> tag inside HTML.
>> Thankfully, this data has its own <script> tag which is appropriately
>> type'd as application/ld+json, so it is not so hard to extract using
>> gb.xml.html and gb.web:
>>
>>    Dim h As New HtmlDocument(your file here...)
>>    Dim el As XmlElement
>>    Dim data As New Collection[]
>>    For Each el In h.GetElementsByTagName("script")
>>      If el.Attributes["type"] <> "application/ld+json" Then Continue
>>      data.Add(JSON.Decode(el.TextContent))
>>    Next
>>
> 
> Thanks, Tobias, for your code. I tried the above code (with the Gambas Components gb.net, gb.net.curl, gb.web, gb.xml, gb.xml.html included: was that overkill?) and it produced a large file named HTMLandXML.txt I've extracted some of it to an attached file named PartOfHTMLandXML.txt However, I have no idea of what to do with it. There are largish blocks of 'code' for each TV programme. Some sample lines which show the (emboldened) data that I want to obtain are:
> line 212:  <h3 class="programme__titles"><a href="*https://www.bbc.co.uk/programmes/m00049tf*"
> lines 214-215:   ><span class="programme__title delta"><span>*Have I Got a Bit More News for You*</span></span><span class="hidden">—</span><span class="programme__subtitle centi"><span>Series 57</span>, <span>*Episode 2*</span></span></a></h3>
> lines 219-220:  <abbr title="*Episode 2 of 9*"><span datatype="xsd:int">2</span>/<span class="programme__groupsize">9</span></abbr>                <span>*Guest host Alan Johnson joins Ian Hislop and Paul Merton for the 
> topical news quiz.*</span>
> I have no idea how to extract this data from the file. Which Gambas Component should I use? Is there an example/tutorial/book on how to extract this data in Gambas? I am a real newbie to HTML & XML etc!
> 
> 
> 
> ----[ http://gambaswiki.org/wiki/doc/netiquette ]----
> 
hello,   here a example;

Public Function GetBBCProgram(Optional solution As Byte, Optional sdate 
As Date) As Collection

  Dim hc As New HttpClient, d As New HtmlDocument, hl As New Collection, 
hkey As String
  Dim res As String, f As XmlElement, n As XmlNode, vdate As String
  If Not sdate Then vdate = Format$(Now, "yyyymmdd") Else vdate = 
Format$(sdate, "yyyymmdd")
  With hc
   .URL = "https://www.bbc.co.uk/iplayer/guide/bbcfour/" & vdate
   .Timeout = 2
   .targetfile = "/tmp/foo.html"
  End With
  res = hc.Download(hc.url)
  'File.Save(hc.TargetFile, res)
  d.HtmlFromString(res)

Select Case solution
Case 1
   shunt:
   Print "1---"
   For Each f In d.Root.GetChildrenByClassName("schedule-item")
   'Print f.GetAttribute("class"), f.type, f.element, f.name, f.value
     Print f.value
     hkey = Left$(f.value, 5)
     hl.Add(Replace(f.value, hkey, ""), hkey)
   Next
   Return hl
Case 2
   Print "2----"
   For Each f In d.Root.GetChildrenByFilter("li[class*=sched]")
     Print f.GetAttribute("class"), f.type, f.element, f.name, f.value
   Next
Case 3
   Print "3--debug---elementnode"
   For Each n As XmlNode In d.Root.AllChildNodes
     If n.Type = 1 Then Print n.Attributes, "!!", n.Name, ":", n.Value
   Next
Case 4
   Print "4--debug---textnode"
   For Each n In d.Root.AllChildNodes
    If n.Type = 2 Then Print n.Attributes, "!!", n.Name, ":", n.Value
   Next
Case Else
   Goto shunt
End Select
End

Public Sub Main()
Dim foo As Collection = GetBBCProgram(DateAdd(Now, gb.Day, 1)) 'tomorrow
For Each item As Variant In foo
   Print foo.key, "!!", item
Next
End

you case use "lynx -dump or w3m -dump" ; bye