R: Partage & Echange

web scraping

Contexte

• Récuperer des données sur internet permet:
  
•       Garder des historiques
  
•       Enrichir ses analyses
  
  

Etude de cas

Problématiques

Qui sont les principaux actionnaires du DAX 30?

Information à retrouver

Information à retrouver

Information à retrouver

Information à retrouver

Information à retrouver

Rappel: Structure page web

Utilisation de {rvest}

• Recherche d’éléments via selecteur HTML
• Recherche d’éléments via selecteur CSS

library(dplyr)
library(stringr)
library(magrittr)
library(rvest)

list_component<- read_html('https://www.marketscreener.com/dax30/company/') %>% 
    html_nodes('.center+ td a') %>% 
    html_attr('href')

head(list_component, 8)

## [1] "/news/latest/Global-Markets-Asian-shares-buoyed-by-China-optimism-markets-eye-U-S-presidential-debate--31367174/"
## [2] "/news/latest/Asian-shares-buoyed-by-China-optimism-markets-eye-U-S-presidential-debate--31367853/"               
## [3] "/news/latest/Stocks-set-for-dire-week-best-for-dollar-since-April--31358570/"                                    
## [4] "/quote/stock/LINDE-PLC-46923083/"                                                                                
## [5] "/quote/stock/SAP-SE-436555/"                                                                                     
## [6] "/quote/stock/SIEMENS-AG-56358595/"                                                                               
## [7] "/quote/stock/ALLIANZ-SE-436843/"                                                                                 
## [8] "/quote/stock/BAYER-AG-436063/"

list_component %<>% 
  str_subset('quote/stock/')

Les 5 plus grands actionnaires de LINDE

x <- list_component[1]
url <- paste0('https://www.marketscreener.com',x,'company/')
read_page <- read_html(url)

# Extraction du tableau
df_stocks <- read_page %>% 
  html_nodes('table .nfvtTab')  %>% 
  .[5] %>% 
  html_table(fill=TRUE) %>% 
  .[[1]]

# Nettoyer le tableau
df_stocks %>% 
  rename(Name=X1, Equities=X2, Hold=X3) %>% 
  filter(row_number()>1) %>% 
  head(5)

##                                                     Name   Equities  Hold
## 1                               The Vanguard Group, Inc. 41,377,684 7.88%
## 2                            SSgA Funds Management, Inc. 21,623,125 4.12%
## 3 T. Rowe Price Associates, Inc. (Investment Management) 20,356,041 3.87%
## 4   Capital Research & Management Co. (Global Investors) 14,540,167 2.77%
## 5                      Norges Bank Investment Management 13,664,164 2.60%

DAX30

• Répeter le processus pour les 30 valeurs

Création de fonction

extract_holders <- function(x){
  url <- paste0('https://www.marketscreener.com',x,'company/')
  
  # Lecture la page
  read_page <- read_html(url)
  
  # Extraction du tableau
  df_stocks <- read_page %>% 
    html_nodes('table .nfvtTab')  %>% 
    .[5] %>% 
    html_table(fill=TRUE) 
  
  # Nettoyage du tableau
  df_stocks[[1]] %>% 
    rename(Name=X1, Equities=X2, Hold=X3) %>% 
    filter(row_number()>1)
}

Utilisation de la fonction

df_all <- tibble(link = list_component) %>% 
  mutate(holders=map(link, extract_holders))

## # A tibble: 30 x 2
##    link                                           holders          
##    <chr>                                          <list>           
##  1 /quote/stock/LINDE-PLC-46923083/               <df[,3] [10 × 3]>
##  2 /quote/stock/SAP-SE-436555/                    <df[,3] [10 × 3]>
##  3 /quote/stock/SIEMENS-AG-56358595/              <df[,3] [10 × 3]>
##  4 /quote/stock/ALLIANZ-SE-436843/                <df[,3] [10 × 3]>
##  5 /quote/stock/BAYER-AG-436063/                  <df[,3] [10 × 3]>
##  6 /quote/stock/DEUTSCHE-TELEKOM-AG-444661/       <df[,3] [10 × 3]>
##  7 /quote/stock/BASF-SE-6443227/                  <df[,3] [10 × 3]>
##  8 /quote/stock/ADIDAS-AG-6714534/                <df[,3] [10 × 3]>
##  9 /quote/stock/MUENCHENER-RUECKVERSICHER-436858/ <df[,3] [10 × 3]>
## 10 /quote/stock/DEUTSCHE-POST-AG-435996/          <df[,3] [10 × 3]>
## # … with 20 more rows

Problématique

• Qui sont les principaux actionnaires du DAX30?

Code associé

df_all_new <- df_all %>% 
  unnest() %>% 
  mutate(montant = str_remove_all(Equities, ',') %>% as.numeric) %>% 
  group_by(Name) %>% 
  summarise(Montant_total=sum(montant)) %>% 
  arrange(desc(Montant_total)) %>% 
  head(10)


ggplot(df_all_new, 
       aes(x=reorder(Name, Montant_total), y=Montant_total)) +
  geom_bar( stat='identity', fill= c('#6891d0',rep('#b2b4b7',9))) +
  geom_text(aes(label=paste(format(round(Montant_total/10^6,0),big.mark=" ", big.interval=3L),' €'), y=(Montant_total-8*10^7)), # color=c( '#b2b4b7','#6891d0'),
            fontface = "bold", color = "#ffffff")  +
  coord_flip() +
  ggtitle('Le gouvernement allemand soutient ses leaders') +
  ylab("Participation au capital des valeurs du DAX30 (en millions d'euros)") + 
  #ylab("Montant (millions de dollars)")
  xlab(" ") +
  labs(caption = "source: www.marketscreener.com", color= 'grey')+
  theme_bw() +
  theme(plot.title = element_text(lineheight = 1.1,size = 20,  hjust = 0.5),
        plot.subtitle = element_text(lineheight = 1.1,  hjust = 0.5),
        plot.caption = element_text(color = "grey", face = "italic"),
        panel.grid.minor = element_blank(),
        panel.grid.major = element_blank(),
        panel.border = element_blank(),
        axis.line.x = element_blank(),
        axis.text.x=element_blank(),
        axis.text.y=element_text(lineheight = 1.5),
        axis.ticks.x =element_blank(),
        legend.position = 'none'
  )

Récapitulatif

{rvest}

• read_html(url)
• html_nodes(code html)
• html_attr(css) (option)
• html_text() ou html_table()

Merci pour votre attention

Questions?