Elevating education -MOHESR Data Warehouse Implmentation
By Imthishan Giado
Charged with determining the education needs of the UAE's students, the Ministry of Higher Education and Scientific Research set out to build a data warehouse which would use business intelligence to put more comprehensive information in the hands of decision makers.
The world of information technology can be enormously discriminatory. When asked to name verticals with the most advanced technology, most people would instantly blurt out sectors like banking or telecommunications.
But it's a little known - and rapidly changing - fact that the higher education sector is fast becoming one of the key implementers of new technology. From campus-wide WiMax networks to advanced e-learning environments with video conferencing - if it's high-tech, you're likely to find it in schools first. Coupled with the fact that many of the largest institutions in the region are publicly-funded and as such, immune at least temporarily from the effects of the global credit crisis and one reaches the conclusion that education IT represents a significant growth opportunity for the industry.
But it's not all roses and sunshine. The rising tide of enrolments in universities and colleges has left the organisations that monitor schools with a bit of dilemma; namely, the volume of data being produced on the students, the courses they take and their academic results require in-depth analysis to be able to detect trends about the future of regional education.
The UAE's Ministry of Higher Education and Scientific Research (MOHESR) was tasked with finding a solution to this problem and opted to build a data warehouse to store data from the nation's educational institutions. This data would then be analysed using business intelligence software to produce decision makers with the facts they needed to best address the needs of the UAE's student population.
Youmna Badowah, director of the higher education planning and coordination department at the Ministry explains the aims of the system: "This whole project was a strategic plan for the UAE ministry of higher education and scientific research to protect and centralise all the data for all the federal and non-federal institutions as well as the higher education institutions. It helps all the decision makers in other federal ministries to have an idea about higher education level."
"It's also to have all the data from the federal and non-federal institutions in one report to make decisions. For the individual institutions, they have their own systems to have the reporting - that's why in previous years, it did not help the decision makers. That's where the idea for the data warehouse came from - to have all the statistics and information in one way for the report," she adds.
For this project, Badowah had more than a little help from Evelyn Babey, senior consultant at the Ministry, who first came over in 2007 for some consulting work and ended up staying to help create the definitions that are so crucial to the workings of a data warehouse. As Babey recalls, the system the Ministry used earlier in 2006 was entirely inadequate for the needs of the both the government and the institutions.
"The office at that point in time actually put together a set of data elements that went out to the federal institutions. They started supplying Excel spreadsheets from September 2006, but they weren't integrated in any way. The Commission for Academic Accreditation (CAA) - which collects the information from the private institutions - began collecting data in 2006. They actually got it into a database over the year because they collect from like 40 or 50 institutions. So they had their set of data, the Ministry had three sets of data on the three federal institutions - and nothing was integrated together," outlined Babey.
In January 2007, the UAE's Strategic Plan was released for all the federal institutions in the country. Badowah says that this is when the project really started to come together, as the Ministry realised how the data warehouse could help fulfil the set goals it had in mind.
"It's also tied to the funding model for the students. You might have heard of problems with the institution for budgeting and that's why students are being rejected from the federal institutions. This is the idea that came up - to have centralised data so that the Ministry of Finance, the Ministry of Presidential Affairs and the three federal institutions can work together on the funding formula and how to connect it to the data warehouse," she states.
Babey explains in further detail: "Originally, the project started out as to build the data warehouse for reporting and research purposes for the UAE and the Ministry. There's a long history going back actually, of the need for some sort of centralised enrolment reporting system. Over the years, it's been expressed in a number of reports that they need something to report data out. What also started coming in were the discussions about whether the higher education was meeting the needs of the government and the people of the UAE in terms of curriculum planning for the labour market needs."
While the initial idea of the data warehouse was floated in early 2007, vendor selection did not begin in earnest until nearly the end of 2007. Badowah says that this delay was down to three reasons; first, the Ministry was occupied with a significant amount of reorganisation. Secondly, government processes mandate a lengthy period of investigation before inviting vendors to make their pitch. Thirdly, all plans have to match the various Strategic Plans of the Ministries involved and receive further approval from the Prime Minister's office - work which consumed the better part of a year.
This is where Babey enters the stage; she was asked to come over from the United States in September 2007 to help with policies and planning during the reorganisation and eventually took over planning for the data warehouse project.
Matters were not helped, says Badowah, by the fact that the Ministry had neither an indigenous IT department nor a budget for the project: "There was no plan - I'm being honest with you - to have a budget for this project. When we started looking at vendors and doing more research about the project, the finance department were working on the budget. According to their meetings with the vendors, they put a project - including the hardware and software - at a value of $681,000."
Vendor selection for the data warehouse began in November 2007. Oracle and Sungard made the final cut for doing a proof-of-concept of their systems. Sungard eventually won the contract and then proceeded with contract negotiations with the ministry - which, as Babey recalls, went on for almost two months due to differences in the language used.
"So we actually signed the contracts in March of 2008 with a delivery date of basically November 1 2008. The driving force behind the timeline then was to be able to report out enrolment for the September 2008 enrolment term for the first time where we actually were going to combine information from all three federal colleges and then follow it with the private colleges, whose data collection and schedules are a little different," she says.
From March to November may strike some as somewhat long for just the implementation stage of a project, but as Babey says, the main obstacle to meeting that target date was the inevitability of the Gulf's holiday season for expats.
The implementation required six consultants from Sungard working in concert with eight individuals from the Ministry's IT team. IR directors also assisted in identifying the data that required for reporting. In total, Babey estimates that there were no less than 20 people working on the project at any particular point in time.
On top of this workforce, Babey also brought some external management help: "I took on the overall coordinating. But we brought in an IT person, Bruce Purcell, from the US to handle the project management because he had experience in data warehousing. He was someone who worked for me previously as a programmer at the University of California Davis and was between jobs,so he came over as a consultant.
He's run data warehouse operations for Stryker Pharmaceuticals in Europe where he worked five years. People here didn't have that experience, so he came over to help with the idea that we'd get it working and then it becomes an industry project."
While some may be surprised that Sungard beat out database giants Oracle for the project, Babey says that the reason the vendor is successful is because its Banner system is already wide in use throughout the three federal ministries. The data warehouse exists as a module within the overall Banner ERP system and as such users were already familiar with its workings. By comparison, selecting Oracle would have required the ministry to virtually build the entire system for the warehouse from scratch, including the various screens and forms required.
This position was confirmed during the proof-of-concept stage, when Babey and Badowah tested Sungard to see if its systems would cope with the Ministry's data.
"We collected Excel spreadsheets from 2006, used 2007 data and deleted all of the identifying information and sent the file to Sungard. They ran it through their database to see how easily it fell into the data elements - and it worked out pretty well. They were able to - on the first try - take what we were feeding them and they went right into the database," explains Babey about the test phase.
"Oracle didn't get the file because they didn't have anything to put it into. Sungard had the database already set, we just forwarded them the information. With Oracle, we would have to build everything, nothing's pre-packaged. Oracle were also more expensive than the package that Sungard had offered at that point. That was certainly something that we looked at but in terms of time, it would have taken a while to develop everything that was in the package that was already tested and proven. The institutions knew what Sungard was about and how to work with it," she confirms.
One of the big issues with projects of this size and scope is support. Sungard's main office is based in Dubai while the Ministries are based Abu Dhabi, but Babey says this isn't a problem since they have extensive service level agreements (SLAs) in place.
Now that the system is up and running, Badowah has had time to reflect and think of future improvements. Unsurprisingly, she says, language is one of them, since Cognos does not support Arabic and the main database administrator is also not a native speaker.
Now that the system's complete, Babey says she is happy that the Ministry has been able to build the system using a detailed project plan and stick to the prescribed dates. However , issues still remain around the question of who within the Ministry has access to the data and what sort of queries they can run.
"Where we are running into difficulty is still getting agreed-upon access, who sees what in regards to confidential information. We have data on students - who's going to have access to that data? Individual institutions can only see their data. What kind of reports can go out publicly - we're dealing with this and working with the colleges and the CAA to come up with an agreement. That's a struggle, because everybody has a different take," she explains.
"We're working on the policies. I'm having a meeting with the institutional research team to take the policies I've already drafted several times - and every time it changes because something else comes up - and try to work through it and come up with a recommendation that we can get to the steering committee. People are getting frustrated - we have all this data now and we really can't share it, even with the people that gave it to us because they haven't agreed on the access," she continues.
"Let's say you're one of the institutions. We have got your information and they need to make sure that their data is being protected. This is one of the challenges but I think that the institutions will in time be comfortable with their data [moving around]. We can have also the responsibility to report data to any federal institutions or decision makers," she adds.
Badowah says that to be truly useful, the data warehouse must eventually link to the wider labour market of the UAE and provide authorities with the crucial data they need to make informed choices about how to educate the workforce.
"According to the UAE Strategic Plan, [the Ministry] are looking for Emiratisation for all the federal ministries and also local institutions and government. With the percentage that they are looking for, if the data warehouse is to help the government for the Emiratisation, we need to ask how we can make this data more useful for the labour market and the people who are working to help the students to find a job. This is one way in which we can help the whole UAE Strategic Plan," she concludes.