Storing data of unknown size in C++

Question

I've been using PHP for about 4 years, however I've come across a problem that requires something with slightly (:P) better performance and so I've chosen C++.

The program I'm writing is a Linux daemon that will scan a MySQL database for URLs to load, load them using cURL, search for a specified string, and then update the database accordingly. The problem that I'm facing is that I don't know the size of the data that needs to be stored in a variable in order to be searched for a specific string.

I had the idea of using a linked list and allocating more nodes as the data fills the list. Is this a good way to go about doing things?

Thanks in advance,

Answer 1

in c++ the vector class can store an unknown sized amount of data.

#include <string>
#include <vector>

std::vector <std::string>Data;

std::string newData = "a String";
Data.push_back(newData);

std::string otherData = "a different String";
Data.push_back(otherData);

of course 'string' could be any data type you want, and you can access the data using the Data[0] to return the first string, and you can use Data.size() to return the amount of strings in the vector/array.

for(int x = 0; x != Data.size(); x++)
{
   //Do what you want with the data here using Data[x]
}

Answer 2

In general, you would want to use one of the standard containers. My personal suggestion is std::vector. YOu can use it as an array (as the data is guaranteed to be contiguous) and it has convenient indexing and insertion operations (seems like you aren't interested in removal at this point).

Specifically, you could set up something like

std::vector<char> buff;
// while you are reading data
buff.push_back (item);

When you are done you can call buff.size to find out how much you read.

As an added bonus (if you are in fact dealing with character buffers), when you finally get all the data you need you can convert to a std::string to do whatever searching you want to do.

std::vector<char> buff;
buff.push_back('e');
buff.push_back('a');
buff.push_back('t');

std::string s(&buff[0], buff.size());

Edited for correctness.

Answer 3

You have a whole lot to discover there.

You should specifically orient your discovery to the STL: use C++ reference

Now you should try to learn how to use:

• std::vector
• std::string
• std::cin and std::cout (globals)

On std::string you should note a good number of algorithm like find, find_first_of, find_last_of.

Note that string manipulations in C++ can be quite challenging (as in verbose).

If you are comfortable with regular expressions, you might be willing to try Boost.Regex. Note that you have to link with its library.

Also, if you come from PHP and want to increase your performance, you could begin with a lot of different scripting languages (Python is my favorite), and it would probably be easier.