Using R to search column in data frame for list of values

I made the jump from using Excel to R for data manipulation when I started on my Master’s project in 2015. I was pretty much forced to – I was looking at datasets of deforestation in the world, with 806,400 pixels. That was already aggregated by a factor of 1000 using Python (i.e. I had 805.4 billion pixels to start with). And I had about 14 such files. For me to load the file in Excel (apart from the fact that it was in raster GeoTIFF format, though it can be converted to a csv file) would already take a while, let alone trying to manipulate it (e.g. add/subtract/find mean etc.). Data manipulation can also be done in Python of course, with the pandas library, but I’m still trying to get my head around Python – maybe once I’ve somewhat mastered data processing in Python I’ll update this post. Here though, I just want to introduce using R for simple data processing, instead of using Excel.

There are many reasons to use R:

  1. It would probably be very useful for your career, if you’re after the kind of jobs that I am. (bit of a circular argument, but really, being proficient in R is very useful especially when many jobs these days require some level of data processing and manipulation). I.e. Looks good on your CV
  2. Coding sounds difficult but once you get a grasp of the syntax, it can and will make life easier (especially when it comes to large amounts of data). I.e. It’s pretty cool in a geeky way, in my opinion, and will save you time

I’m obviously no R expert, nor am I here to create tutorials for R beginners, I just wanted to write about bits of R programming that makes me go ‘that’s awesome’, or #:D 😀 😀 in my R script. As well as maybe help people trying to make the transition to using R as their main programme of choice for data analysis.


I wanted to find out if a list of countries are in this dataset that I have. What one could do in Excel would be to Ctrl+F the dataset for each of the countries on the list (or maybe there’s a more efficient way about it, I’ve stopped using Excel a while ago to learn the new techniques, but please comment if you know of something!). I only have 10 (the member states of the Association of South East Asian Nations), so that wouldn’t be too painful. But imagine you have a list of 100 countries – while the digital age and Microsoft Excel have made life easier for us, saving us from having to pore over pieces of paper – that quickly becomes tedious work.

[code language=”r”]
# I’m not attempting to teach basics of R here, if you are unfamiliar with how to load files, please do a Google search. Here, I am using a GIS shapefile, though the same can be applied for a normal csv file
# Obtain a shapefile of world boundaries
download.file("http://thematicmapping.org/downloads/TM_WORLD_BORDERS-0.3.zip", destfile="TM_WORLD_BORDERS-0.3.zip")
unzip("TM_WORLD_BORDERS-0.3.zip", exdir=".")

# I am using rgdal library to read in this shapefile
library("rgdal")

# read in the GIS file
world_border <- readOGR(dsn=".", "TM_WORLD_BORDERS-03")
# you can look at the data contents (i.e. attribute table)
world_border@data #prints everything, I just want to look at the first few entries
head(world_border@data)

# at this point, I can ‘manually’ search the GIS files for the countries I want
grep(‘Singapore’, worldbound@data$CNTRY_NAME)
# and do the same for all the countries I’m interested in

# alternatively, I can create a list of the SEA countries
SEA <- c(‘Vietnam’, ‘Laos’, ‘Cambodia’, ‘Thailand’, ‘Myanmar’, ‘Malaysia’,
‘Indonesia’, ‘Singapore’, ‘Philippines’, ‘Brunei’)
# and create a loop to search the world boundaries file for these countries, and only report an error if the country can’t be found
for (i in 1:length(SEA)){
if (length(grep(SEA[i], worldbound@data$CNTRY_NAME))==0)
stop(paste(‘unable to find’, SEA[i], sep=" "))
}
# I had initially included ‘East Timor’ in the list of SEA countries, and my loop returned ‘unable to find East Timor’, so I know it works
# this would also work with a csv file
[/code]

I did this search not so much to check if these countries exist on the world map, but more to check on the spelling of the countries. There are some databases which spell ‘Viet Nam’ instead of ‘Vietnam’, for example, and if I don’t check how it’s spelt, I might run into errors later on. I don’t presume to be an expert in R, and this code might be considered clunky by more proficient R coders, but this is the best I could come up with!

Apart from using Excel to Ctrl+F the countries I want, I could also have used QGIS to load the file and click on each country for information, or use a polygon to highlight all the countries in the region and obtain the information at once. Or used Python, or any other programme that allows data management. I just like R, and though I could easily and perhaps more quickly have done it in QGIS (with its GUI, without having to Google for the right R code to do what I want, because I’m still not all that familiar with R), I enjoy these little challenges, and it’ll eventually make me more proficient.

Doing fieldwork as a female

How do females deal with their menstrual period while out field? It is a rather underrated and neglected topic, so today, we had a female health in the field session for the (female) students in NTU’s Asian School of the Environment. Though I don’t teach the students, being a Singaporean who’s used a menstrual cup for the last 6 years on field trips, diving trips, and hiking and mountain-climbing expeditions gave me an insight with useful experiences and stories to share.

The issues surrounding doing fieldwork as a female may not be obvious to males (who are also the more common sex in the field), and many females might feel like menstruation is a taboo topic. But getting menstrual periods is something that happens to half of the world’s population and just needs to be dealt with appropriately. Just as many people with the means to have embraced the use of sanitary pads and so go about their everyday life, instead of being confined to their homes while they’re having their period, we shouldn’t feel like we can’t do field work (i.e. go outdoors without access to a toilet with running water) while we’re having our periods, particularly when alternative possibilities exist.

The session started out with Urinary Tract Infections (UTIs) and vaginal thrush, and how to prevent them. It seems like the best way would be to not hold your pee (wild pees ftw!), drink lots of water, and maintain a basic level of hygiene. Girls do require a higher minimum level of hygiene than boys do I think, going by my experience thru-hiking in Chile. Using toilet paper (and burying it) and changing fresh underwear is almost a necessity for us, whereas guys don’t seem to get UTIs when they neglect those. Most people are aware of feminine hygiene though, I think.

The more pertinent issue was that of dealing with periods while outdoors. Apart from not having a ready bin for disposal (i.e. you have to bag your used sanitary napkins/tampons and bring it with you till you reach ‘civilisation’), the lack of a toilet bowl/running water can sometimes deter females. Personally, the massive waste of single-use, disposable but un-recyclable sanitary pads/tampons are a major con for using them, and after experiencing the comfort of using a menstrual cup as well, I would never use the disposables again (except for emergencies/supplement to prevent leaks!). I switched from using sanitary pads to using a menstrual cup in 2011 when I planned to hike in Nepal, and have since used it even while camping in Australia, climbing Mt Kilimanjaro in Tanzania, on expeditions in Ecuador/Peru, field trips in the UK, and long-distance hiking in Chile.

Menstrual cups are great especially when you’re out in the field because

  1. You don’t have to deal with disposing bloodied pads/tampons, you can just empty the blood out where you’ve been peeing/pooping (buried if necessary!).
  2. You don’t have to bring a massive supply of disposables.
  3. You can swim/dive.

There are many other pros of using menstrual cups over pads or tampons, like

  1. It’s really comfortable. You don’t end up sitting in your warm pool of blood, as you do with a pad.
  2. You save a lot of money not having to buy pads/tampons. I do use a thin, light-flow pad on my heavy flow days just in case my cup leaks because I haven’t had time to empty it, but otherwise a panty liner will suffice, so I end up using just 2-4 pads over the course of my period, as opposed to at least 10. And a menstrual cup can be used for many, many years – I’m still using the same one.
  3. You have to go to the toilet less often. When I was using a pad, I often worried about leaking, and so kept going to the toilet to check. The menstrual cup just needs emptying about twice a day usually, thrice if it’s a heavy flow, and once when it’s the last day.
  4. They don’t cause Toxic Shock Syndrome, because the cup merely collects the blood instead of absorbing it like tampons. Hence you can keep it in for longer.
  5. You’re not contributing to unnecessary waste. So much plastic and other waste goes to our landfills/incinerators every year. Waste less, don’t use disposables.

There is a higher starting up cost to using menstrual cups, in that they cost between S$30 (for Freedom Cups, a Singapore-based company) to S$50 (for UK-based Mooncup, which is what I’m using and was the only option in Singapore 6 years ago). It also took me about two periods to get familiar and comfortable with putting the cup in and taking it out.

However, maintaining hygiene while using the Mooncup is not difficult, I wash it out with water and put it back in (using my water bottle if a tap is not available). I have even accidentally dropped it in the toilet bowl while cleaning it out, and just rinsed it and popped it back in with no further issue (I am not advising cleaning your menstrual cup with toilet bowl water though, especially after you’ve peed in it…).

Everyone would have their own way of dealing with periods while outdoors, but to me, menstrual cups are a genius invention that should be embraced. It would be a huge shame if menstruation was a deterrent to more girls doing field work or going outdoors, though having (really bad) menstrual cramps would be a completely different issue.

We may be physically weaker (for most of the population) and have a slightly greater need of personal hygiene than males, but we shouldn’t fear it to be an impediment when doing fieldwork, nor allow it to be an excuse for not having more females on an expedition, because there is a need and value for women to be in the field at times, as Haifaa Abdulhalim accounts as when working as IUCN’s World Heritage Coordinator in the Arab region.

[Update: 10 May 2017] There are several other slightly more (than just trying to get let the blood out in a less disruptive fashion) drastic options for controlling your period, e.g. birth control/contraceptive pills, and intrauterine device (IUD). They both have the main/side effect of preventing pregnancy, so depending on your personal beliefs, you may prefer to use/avoid them. I’m not as familiar with these options so did not mention them as methods for controlling period bleeds, but from comments provided, they are sometimes a better option than menstrual cups.