Discussion about this post

User's avatar
George's avatar

Fantastic video!

I really like the music but an not familiar with it.

Please provide source.

Never mind. Just reread the article.

Getting a Ph.D lowered my reading skills.

Expand full comment
Bacca400's avatar

Wow! That's a lot of work!

> write a Python script to use Selenium in order to webscrape the titles of the papers from the links.

OMG. Programmer here. Isn't there any metadata in some webpages to get the title of the study, or some other data about the study?

I'd be willing to help you with programming tasks, I just use Perl. It's great at text processing. For example I can deduplicate studies as I've done deduplication of mailing addresses. What a task that was!

Can't you deduplicate by the DOI number or DOI link? Every study has one I believe and the DOI number is unique to a study. More info at https://doi.org

What about having several people make a growing spreadsheet of these articles with 1 row per study and different fields for data? Such as study name, 3 main researchers (sometimes there are 10 authors), DOI number, DOI link, publish date, link to full study, link to abstract only (not all full studies are free), and 3-5 topics for each study. That's just off the top of my head without doing any analysis on the study data.

Expand full comment
15 more comments...

No posts