Score:0

Is there any trick to collecting data from a web page that using JavaScript and setting current time and date automatically?

in flag

I work under Ubuntu Linux in the terminal, using shell script wget, httrack and:

google-chrome --headless --dump-dom 'https://www.pref.kanagawa.jp/sys/taikikanshi/kanshi/realtime/index.html' > ~/file.html

I want to extract these lines hourly from the above site:

< tr class="tr_line_1">
< td class="le" id="station1" headers="stationTop" height="40px" >神奈川県< /td >
< td class="le" id="station1" headers="stationName" height="40px" >鎌倉市役所< /td >
< td class="le" headers="station1 stationDiv" height="40px">一般局 < /td >
…
< td headers="station1 WD WDUnit" height="40px">北東 < /td >
< td headers="station1 WV WVUnit" height="40px">1.2 < /td >

(I want to understand the sea breeze conditions near the shores of Sagami Bay. I am a sailing fan and using these data for the private understanding of regattas only.)

Thank you for any advice!

pLumo avatar
in flag
you can do such things with `python` and `selenium` modul. However, as this is more a programming than an ubuntu question, I guess you should search for an answer / or ask at [stackoverflow.com](http://stackoverflow.com).
bac0n avatar
cn flag
... [scrapy](https://stackoverflow.com/tags/scrapy)
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.