For charities, fielding ‘big data’ is an onerous task—one that is ongoing and requires combing through a vast and complex amount of online data. Fed up with this process, one organization is taking steps to make data do all the grunt work.

Third Sector Publishing has teamed up with a software engineering team from Lakehead University to develop a system that will help compile and automate ‘big data.’ The goal of the project is to automate the content search requirements of CharityCAN, the organization’s flagship online resource.

“In terms of improving CharityCAN, this project aims to gather as much useful data as possible and then to make it as relevant and accessible as possible,” says Anderson Charters, President of Third Sector Publishing. “As data gets updated elsewhere, it will be extracted and published on CharityCAN in as automated a fashion as possible.”     

CharityCAN provides in-depth information about Canada's 85,000-plus registered charities, and private and public foundations. Among other data sets, information is based on the charities’ last five years of financial statements.

The ‘big data’ project aims to broaden the scope of data being gathered—in turn strengthening the CharityCAN resource in the following ways:

• For fundraisers, enhancing wealth identification information and prospect research.
• For donors, enhancing accessibility of information including financial metrics, mission details and administrative costs.  

Building a knowledge-based system

Under the direction of Dr. Rachid Benlamri, professor of Software Engineering at Lakehead, the research team identified three challenges to overcome:

• Ability to extract data. Data comes in different forms (.htm, .pbs, .xml, etc.) and this is a challenge when extracting data through an automated process.  
• Ability to capture business analysis. Capturing semantic relationships between roles in a business.  
• Ability to keep data current. As new information becomes available, it is extracted and updated automatically.

Meeting these challenges, the team was able to develop a knowledge-based system that integrates data from different sources. As this was the first phase of the project, next steps will focus on making the results operational.

The project was funded in part by the Ontario Centres of Excellence, which sees this project as just the first step in what could lead to bigger projects in the near future. Now that the first phase is complete, Charters is looking forward to moving on to the second and third phases with the Lakehead team.

“We are extremely fortunate to have the opportunity of working with Dr. Benlamri and his research team,” said Charters. “The software and information technology developed from this project will help CharityCAN become more effective and solidify its leading position in the nonprofit prospect research market.”

Michelle Jondreau is a communications professional with an avid interest in all things HR. On top of that, she hails from the nonprofit world and as such has a keen understanding of sector trends and issues. Follow her @majondreau.