Was Brazil’s performance in poverty alleviation an extraordinary one under Lula’s and Dilma Rouseff’s administrations? Looking into data from the World Bank’s API with Python.

Dr. rer. pol. Eduardo W. Ferreira (Ph. D.)

Introduction

The arrest of former Brazil’s President Luiz Inacio Lula da Silva due to corruption and money laundering crimes have disappointed many of his supporters. This and other arrests of prominent politicians and business leaders have renewed hopes that rule of law was gradually starting to apply to the powerful.

Lula still remains a wildly popular leader whose administrations were credited with bringing millions out of poverty in one of the world’s most unequal countries1. Some of Lula’s supporters have insistently argued that without his government administrations (and “almost” two other ones of his successor), poverty alleviation would not have taken place. They often portrait Brazil’s performance in this regards as something unique in the world and dependent mainly on Lula. Following this argument, many supporters have claimed political persecution of Lula by elites that now could not bear travelling in airplanes together with the poor who thanks to Lula became able to afford it. Such strong assumptions seem exaggerated.

Matt Ridley’s best seller The Evolution of Everything: How New Ideas Emerge, for example, explains very nicely that:

Change in technology, language, morality, and society is incremental, inexorable, gradual, and spontaneous. It follows a narrative, going from one stage to the next, and it largely happens by trial and error – a version of natural selection. Much of the human world is the result of human action but not of human design: it emerges from the interactions of millions, not from the plans of a few.

So, I thought that a fresh look into the newest poverty indicators could help avoiding people falling into exaggerated claims by politicians in Brazil and in other countries.

The tools

Although there are many analyses indicating that poverty reduction has been a global phenomenum, I wanted to investigate this myself directly from World Bank’s datasets using Python. The examined indicator was the poverty headcount ratio at $1.90 a day (2011 PPP) (% of population) from 2000 to 2016. This is the share of the population living in the bottom of the pyramid.

For that, I employed a range of libraries including Wbdata. This is a great python interface to find and request information from the World Bank’s various databases, either as a dictionary containing full metadata or as a pandas DataFrame. Currently, wbdata wraps most of the World Bank API, and also adds some convenience functions for searching and retrieving information.

The code chunks and output below present all the steps for downloading, cleaning and visualising data. Hash signs (“#”) indicate my explanatory comments to allow full auditing of results and reproducibility.

The results

The plot above and results below do not suggest that the reduction in terms of poverty headcount ratio at $1.90 a day (2011 PPP) (% of population) was a particularly astonishing one in Brazil. Most importantly, this was a global phenomenum. It would be very surprising if Brazil had not reduced the share of its population living under extreme poverty. Hence, the argument that the verified poverty alleviation is something unique of Brazil and that it should be a reason for pardoning crimes of corrupt politicians seems to be a weak one. However, it can be useful in an election year as 2018 in an unequal and politically polarised country such as Brazil.

Source code

For transparency and reproducibility, all the steps taken to generate the plot using Python 3.6.4 are presented below. Suggestions for improvement are always welcome! Have fun, keep coding and vote wisely.

# Imports required libs

import wbdata 

import pandas as pd

from pandas.io.json import json_normalize

from datetime import datetime

from pandas import Series, DataFrame

import numpy as np

import matplotlib.pyplot as plt

import scipy 

# Reads data as JSON 

## Poverty headcount ratio at $1.90 a day (2011 PPP) (% of population)

SIPOVDDAY = wbdata.get_data(“SI.POV.DDAY”)

# Converts data as dataframe

df =json_normalize(SIPOVDDAY)

# Selects columns of interest

povdata = df[[“date”, “country.value”, “value”]]

# Converts years to integer numebrs and subsets data as from 2000

povdata = povdata[povdata[“date”].astype(np.int)>=2000]

# Change variable type to float

povdata[“value”] = povdata[“value”].astype(np.float)

#Renames columns

povdata.columns = [u’date’, u’country’, u’value’]

# Drops regional data

povdata = povdata[~povdata[‘country’].isin([“Arab World”, “Caribbean small states”, “Central Europe and the Baltics”,

                                 “Early-demographic dividend”, “East Asia & Pacific”, 

                                  “East Asia & Pacific (excluding high income)”, 

                                  “East Asia & Pacific (IDA & IBRD countries)”, “Euro area”, 

                                  “Europe & Central Asia”, “Europe & Central Asia (excluding high income)”,

                                  “Europe & Central Asia (IDA & IBRD countries)”, “European Union”,

                                  “Fragile and conflict affected situations”,

                                  “Heavily indebted poor countries (HIPC)”, “High income”, “IBRD only”,

                                  “IDA & IBRD total”, “IDA blend”, “IDA only”, “IDA total”, “Late-demographic dividend”,

                                  “Latin America & Caribbean”, “Latin America & Caribbean (excluding high income)”,

                                  “Latin America & the Caribbean (IDA & IBRD countries)”, 

                                  “Least developed countries: UN classification”,

                                  “Low & middle income”, “Low income”, “Lower middle income”, 

                                  “Middle income”, “High income”, “Middle East & North Africa”,

                                  “Middle East & North Africa (excluding high income)”,

                                  “Middle East & North Africa (IDA & IBRD countries)”, “North America”, 

                                  “Not classified”, “OECD members”, “Other small states”, 

                                  “Pacific island small states”, “Post-demographic dividend”,

                                  “Pre-demographic dividend”, “Small states”, “South Asia”,

                                  “South Asia (excluding high income)”,

                                  “South Asia (IDA & IBRD)”, “Sub-Saharan Africa”,

                                  “Sub-Saharan Africa (excluding high income)”,

                                  “Sub-Saharan Africa (IDA & IBRD countries)”, “Upper middle income”, “World”

                                 ])]

The dataset spans a period of 18 years with data for 217 countries as presented in the summary table below (see row “unique” for column “country”). Not all countries have data for all years (see pivot table in the end of this notebook).

# Describes clean dataset

povdata.describe().transpose()


count mean std min 25% 50% 75% max
value 717.0 3.785216 6.451344 0.0 0.1 0.7 4.5 42.2

# Filters dates after 2000 and drops NaNs (missing data)

povdata = povdata[povdata[“date”].astype(np.int)>=2000].dropna()

The table below describes the dataset once we have removed missing data and subset only countries with at least 10 years of data as from 2000. From the 217 countries, only 51 remain in the dataset. Brazil has 14 years of data available for the indicator of interest / Poverty headcount ratio at $1.90 a day (2011 PPP) (% of population).

# Drops NAs for values and counts remaining observations by country 

subsetpovdata = povdata.dropna(subset=[‘value’]).groupby(‘country’).count()

# Subsets only those countries with at least 10 years of data 

subsetpovdata = subsetpovdata.loc[subsetpovdata[“value”]>10]

# Filters dataset based on subset

povdata = povdata.loc[povdata[“country”].isin(subsetpovdata.index)]

# Prints shape of dataset

povdata.describe().transpose()


count mean std min 25% 50% 75% max
value 717.0 3.785216 6.451344 0.0 0.1 0.7 4.5 42.2

The table below provides summary statistics for Brazil’s poverty headcount ratio at $1.90 a day (2011 PPP) (% of population).

# Subsets data for Brazil only

povdatabr = povdata[povdata[“country”]==”Brazil”].sort_values(by=[‘date’])

# Implements linear interpolation to fill in missing data

povdatabr = povdatabr.interpolate(method=’linear’, axis=0).ffill().bfill()

# Describes data for Brazil

povdatabr.describe().transpose()


count mean std min 25% 50% 75% max
value 18.0 6.569444 3.136368 2.8 3.8 5.5 9.425 11.6

This is the ultimate plot that I was looking for. Brazil indeed experienced a reduction in the poverty headcount ratio at $1.90 a day (2011 PPP) (% of population). It started from a better point than many countries but the slope of the line does not seem extraordinary when compared to other countries.

# Creates empty plot

fig = plt.figure(); ax = fig.add_subplot(1,1,1)

labels = ax.set_xticklabels(povdata.pivot(“country”, “date”, “value”).columns, 

fontsize = “small”, rotation = 45)

# Plots values for all countries with available data for the period

ax.plot(povdata[“date”], povdata[“value”], “k–“, 

        label = “Other countries”, color = “y”, alpha=0.7)

# Plots values for Brazil

ax.plot(povdatabr[“date”], povdatabr[“value”], “k”, 

        label = “Brazil”, color = “green”, marker = “o”)

# Includes title, legend and labels 

ax.set_title(“Poverty headcount ratio at $1.90 a day (2011 PPP) (% of population)”)

ax.legend(loc = “best”, frameon = False)

ax.set_xlabel(“Years”)

ax.set_ylabel(“% of population”)

# Draws vertical line and annotation indicating start of Lula’s presidency

plt.plot([2003, 2003], [0, 11.1], ‘k–‘, lw=1)

ax.annotate(‘Start of Lula\’s presidency’, xy=(2003, 35), xytext=(2003, 35),

            rotation = 90, fontsize = “small”, alpha = 0.7)

# Draws vertical line indicating end of Lula’s presidency and start of Dilma’s

plt.plot([2011, 2011], [0, 4.7], ‘k–‘, lw=1)

ax.annotate(‘Lula\’s succession by Rouseff’, xy=(2011, 31), xytext=(2011, 31),

            rotation = 90, fontsize = “small”, alpha = 0.7)

# Draws vertical line indicating end of Dilma’s presidency

plt.plot([2016, 2016], [0, 3.4], ‘k–‘, lw=1)

ax.annotate(‘End of Rouseff\’s presidency’, xy=(2016, 29), xytext=(2016, 29),

            rotation = 90, fontsize = “small”, alpha = 0.7)

The pivot table below presents the original values behind this one plot above. Missing data for Brazil has been filled in by linear interpolation in the plot above. That is the reason why the series for Brazil in the plot above also contains data for 2000, 2010, 2016 and 2017 although in the table below it appears as missing data (NaN).

# Pivots data

povdatapivot = povdata.pivot(“country”, “date”, “value”)

# Prints pivot data

povdatapivot

date 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
country
















Argentina 5.7 9.4 14.0 7.0 5.4 3.9 3.3 2.9 2.6 2.6 1.1 0.9 0.8 0.8 0.7 NaN 0.6
Armenia NaN 19.3 15.1 11.4 7.9 4.4 3.2 2.8 1.4 1.9 1.9 2.2 1.6 2.2 2.3 1.9 1.8
Austria NaN NaN NaN 0.2 0.0 0.2 0.0 0.5 0.5 0.5 0.5 0.5 0.5 0.2 0.2 0.7 NaN
Belarus 7.1 3.2 2.0 1.4 0.3 0.4 0.1 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Belgium NaN NaN NaN 0.2 0.0 0.2 0.2 0.2 0.0 0.0 0.2 0.2 0.2 0.2 0.0 0.0 NaN
Bolivia 28.6 22.8 24.7 NaN 13.7 19.3 16.4 12.4 11.1 10.5 NaN 7.3 8.2 6.9 5.8 6.4 7.1
Brazil NaN 11.6 10.3 11.1 9.7 8.6 7.2 6.8 5.6 5.4 NaN 4.7 3.8 3.8 2.8 3.4 NaN
Colombia 16.4 19.7 14.3 12.0 10.9 9.7 NaN NaN 10.4 9.0 7.8 6.4 6.3 5.7 5.0 4.5 4.5
Costa Rica 6.5 4.7 4.5 4.4 4.3 3.1 3.2 1.7 2.2 2.4 1.5 1.6 1.6 1.6 1.4 1.5 1.3
Cyprus NaN NaN NaN NaN 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 NaN
Czech Republic NaN NaN NaN NaN 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 NaN
Denmark NaN NaN NaN 0.2 0.2 0.2 0.2 0.0 0.2 0.5 0.0 0.0 0.0 0.2 0.2 0.2 NaN
Dominican Republic 5.5 4.0 6.0 6.9 8.8 5.8 4.5 4.5 3.8 3.3 2.6 2.9 2.7 2.4 2.3 1.9 1.6
Ecuador 28.2 NaN NaN 14.5 15.0 12.1 8.1 8.5 7.5 7.2 5.6 4.7 4.5 3.2 2.6 3.4 3.6
El Salvador 12.2 13.5 14.0 14.8 10.9 10.4 6.9 4.5 6.7 6.4 5.5 4.5 4.1 3.2 3.0 1.9 2.2
Estonia NaN NaN NaN 2.5 1.0 0.7 0.7 0.5 0.5 0.5 0.7 1.0 0.7 1.0 0.7 0.5 NaN
Finland NaN NaN NaN 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 NaN
France NaN NaN NaN 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 NaN
Georgia 21.0 20.9 11.5 11.6 11.3 12.1 11.6 13.3 12.4 10.4 13.3 11.6 9.0 6.9 5.3 4.0 4.2
Greece NaN NaN NaN 0.5 0.7 0.2 0.5 0.5 0.5 0.5 1.0 1.2 1.5 1.0 1.5 1.5 NaN
Honduras NaN 20.6 23.4 28.1 27.2 26.5 22.5 17.2 16.5 13.8 15.0 17.1 19.6 17.3 15.9 16.2 16.0
Hungary NaN NaN NaN NaN 0.2 0.7 0.2 0.0 0.0 0.0 0.0 0.0 0.2 0.2 0.7 0.5 NaN
Iceland NaN NaN NaN 0.0 0.2 0.0 0.2 0.0 0.0 0.0 0.2 0.2 0.0 0.0 0.0 NaN NaN
Indonesia 39.3 35.5 23.0 22.6 23.9 21.1 27.4 22.5 21.4 18.2 15.7 13.3 11.7 9.4 7.9 7.2 6.5
Ireland NaN NaN NaN 0.2 0.0 0.0 0.1 0.0 0.2 0.5 0.5 0.5 0.2 0.7 0.5 NaN NaN
Italy NaN NaN NaN 0.7 0.7 0.7 0.7 0.7 0.7 1.0 1.2 1.2 1.2 1.4 1.2 NaN NaN
Kazakhstan NaN 10.5 6.9 4.6 2.3 6.9 0.6 0.3 0.1 0.2 0.1 0.0 0.0 0.0 0.0 0.0 NaN
Kyrgyz Republic 42.2 36.4 34.2 28.1 13.6 15.4 9.9 9.9 4.0 2.1 4.1 1.8 2.9 3.3 1.3 2.5 1.4
Latvia NaN NaN NaN NaN 2.5 2.2 1.5 0.5 1.2 1.7 1.7 1.0 1.0 1.2 0.7 0.7 NaN
Lithuania NaN NaN NaN NaN 2.7 1.7 1.5 1.2 1.5 2.2 1.5 0.7 1.0 0.7 1.2 0.7 NaN
Luxembourg NaN NaN NaN 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.2 NaN NaN
Moldova 35.9 27.6 16.6 7.6 8.7 14.0 2.4 1.1 1.3 1.0 0.5 0.3 0.3 0.1 0.0 0.0 0.2
Netherlands NaN NaN NaN NaN 0.2 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.0 0.0 0.0 0.0 NaN
Norway NaN NaN NaN 0.2 0.2 0.2 0.2 0.2 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.2 NaN
Panama 12.4 15.5 11.2 11.2 10.2 10.0 10.5 7.8 7.0 3.2 4.5 3.0 4.2 2.8 3.5 2.0 2.2
Paraguay NaN 8.9 13.2 8.3 5.7 6.1 7.9 7.8 4.3 5.8 5.5 5.0 3.3 1.8 2.4 1.9 1.7
Peru 16.4 17.1 15.1 11.9 13.6 15.5 13.5 11.1 8.9 7.0 5.5 5.2 4.7 4.3 3.7 3.6 3.5
Poland 0.1 0.1 0.1 0.0 0.1 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 NaN
Portugal NaN NaN NaN 1.0 0.5 0.5 0.0 0.2 0.2 0.2 0.2 0.5 0.7 0.7 0.5 0.5 NaN
Romania 7.5 5.5 5.4 3.9 2.8 2.1 2.4 1.3 0.0 0.0 0.0 0.0 0.0 0.0 NaN NaN 0.0
Russian Federation 2.2 1.3 0.7 0.8 0.7 0.5 0.3 0.1 0.1 0.0 0.1 0.0 0.0 0.0 0.0 0.0 NaN
Serbia NaN NaN 0.2 0.5 0.9 0.9 0.5 0.4 0.1 0.1 0.2 NaN NaN 0.3 NaN 0.1 NaN
Slovak Republic NaN NaN NaN NaN 0.0 0.2 0.2 0.2 0.5 0.4 0.5 0.5 0.2 0.2 0.7 0.7 NaN
Slovenia NaN NaN NaN NaN 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 NaN
Spain NaN NaN NaN 0.7 0.7 0.7 0.7 0.4 0.5 0.7 0.7 1.2 1.0 1.2 0.7 1.0 NaN
Sweden NaN NaN NaN 0.2 0.5 0.9 0.2 0.2 0.5 0.2 0.2 0.5 0.2 0.5 0.5 0.5 NaN
Thailand 2.5 NaN 1.1 NaN 0.8 NaN 0.7 0.3 0.1 0.2 0.1 0.0 0.1 0.0 NaN NaN NaN
Turkey NaN NaN 1.7 3.7 2.1 2.6 1.9 1.4 0.6 0.9 0.8 0.3 0.3 0.3 0.3 0.3 0.2
Ukraine NaN NaN 1.7 0.6 0.6 0.3 0.1 0.1 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.1 0.1
United Kingdom NaN NaN NaN NaN 0.5 0.5 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 NaN
Uruguay NaN NaN NaN NaN NaN NaN 0.5 0.3 0.2 0.2 0.1 0.1 0.1 0.2 0.1 0.1 0.1
  1. SAVARESE, M. & PRENGAMAN, P. (2018) “Brazil Is Bracing for the Arrest of Former President Luiz Inacio Lula da Silva”, Times Magazine, 6 April 2018. Available at: https://time.com/5230915/brazil-former-president-lula-da-silva-arrest-warrant/