Tutorial: using cURLcpp to Create a C++ Web Reader in Ubuntu

  • Posted on: 8 March 2015

In this article, we're going to create a C++ application with cURLcpp that reads a webpage and prints the HTML source to stdout.

I wanted to try cURLcpp as opposed to the tried and usually true cURLpp, mostly for fun.

The bulk of the article is really a "for-dummies" guide to setting up your environment to do so. I won't really go into detail about how to use cURL or cURLcpp. In fact, we're just going to compile the first example from the cURLcpp readme. Once you've got a good cURLpp development 'environment' setup, it shouldn't be too hard to cut and try different cURLpp features and code.

Prereqs:

Before going further, I'm going to assume you're on a Linux-based system (I'm in Ubuntu, but the experience should be similar for other flavors of Linux). I'm also going to assume you might have done some C++ development before, which means you have gcc installed and updated, and you have make installed and updated.

Tooling-up:

We're going to add git and cmake to our environment, so if you already have these, feel free to skip this section.

First, we're going to install git. Git is a source control system like CVS or SVN. If that's not ringing any bells for you, don't worry. For our purposes, we're just going to use it to grab some code from github. This is sort of optional, but it's my preferred way of getting things from github, which is where cURLcpp lives. Open a terminal and go get git:

sudo apt-get install git

Next we need cmake. cmake is a tool that let's one describe a project's build process in a platform indepedent way. For example, if we have a cmake input file, we can ask cmake to give us makefiles if we're on Linux, or a Visual Studio project if we're on windows. Install this with:

sudo apt-get install cmake

Getting cURL:

cURLcpp is a wrapper for cURL (really, libcurl), which is a library written in C. At the end of this tutorial, you will have a C++ application that reads a webpage from the internet, but behind the scenes, cURL will still be doing a lot of the heavy lifting. We need to download the development libraries and headers -- not just the application. Get all this stuff with:

sudo apt-get install libcurl4-openssl-dev

Note: There are other versions of libcurl that use stuff besdies openssl, I chose the openssl version for really no reason at all.

Okay, now we have the cURL libraries (which we can link against) and some cURL headers. We have everything we need had we wanted to write a C application.

Getting and Building cURLcpp:

Now we're going to get cURLcpp.

Before we do this, create a directory in your filesystem called "curl_tut". Now enter that directory and create two more directories, "curlcpp", and "my_curl_app". We'll use the former now, and the latter later.

Go ahead and enter the directory "curlcpp", open a terminal, and run the following:

git clone https://github.com/JosephP91/curlcpp

Here, we're using git to clone the source code onto our local machine for use. git can do much, much more, but this is the extent of our use for now.

Alright, now to build. You should still be in the "curlcpp" directory. Now, we're going to run cmake to generate some makefiles for us:

cmake -G "Unix Makefiles"

You should now have a file called "Makefile" in the "curlcpp" directory. Now build the project:

make

This will spit out a static library that we can link against in curlcpp/src: libcurlcpp.a

Building the Example:

Ok, now find your way into the "my_curl_app" directory.

Create the follwing cpp file, and name it "main.cpp" This is actually the example directory from the curlcpp README file:


#include "curl_easy.h"

using curl::curl_easy;

int main(int argc, const char **argv) {
    
    curl_easy easy;
    
    easy.add(curl_pair<CURLoption,string>(CURLOPT_URL,"http://www.google.it") );    
    easy.add(curl_pair<CURLoption,long>(CURLOPT_FOLLOWLOCATION,1L));
    
    try {
        easy.perform();
        
        
    } catch (curl_easy_exception error) {
        // If you want to get the entire error stack we can do:
        vector<pair<string,string>> errors = error.what();
        // Otherwise we could print the stack like this:
        error.print_traceback();
        // Note that the printing the stack will erase it
    }
    return 0;
}

Now we also need to create a CMakeLists.txt file that will help us build our project. Well, actually we don't NEED to, but I find them easier to work with than makefiles:


cmake_minimum_required(VERSION 2.8)
set(CURL_MIN_VERSION "7.28.0")

# Setting up project
project(my_curl_app)

#set c++ flags
set(CMAKE_CXX_FLAGS "-g -Wall -std=c++11")

#set directories
include_directories(../curlcpp/include)
link_directories(../curlcpp/src)

add_executable(my_curl_app
main.cpp
)

#link curl and curlcpp
target_link_libraries(my_curl_app libcurlcpp.a curl)

Now we're going to run cmake and make similar to the way we did to build curlcpp. First, run:

sudo apt-get cmake -G "Unix Makefiles"

cmake might give you some business about using relative paths. Let it go for now. Then run make:

make

Ok, if everything went well, you should have your application built. Run the application:

./my_curl_app

You should see the source of www.google.it spit out in your terminal window. Success!