BI Project Management Tasks
Project
Project manager
Project artifacts
ID
BI Activity
In each row, list the specific task to be completed in
order to complete a BI Solution from Start to Finish?
1
2
3
4
5
6
7
8
9
10
11
12
13
BI Activity Purpose
Describe the purpose of the BI task
Owner/Responsible
Stakeholder
Who will own and
complete the BI
activity?
Status
Date
Entered
InDate the
Progress,
activity
Closed, or
was
Pending Entered in
Project
Plan
BI Project Management Tasks
14
15
Project #
Sponsor
Updated
Planned
Completion
Actual
Completion
Date the
activity is
planned to be
completed
Date the
activity was
completed
services
A proven path to success
Use Teradata Professional Services to guide your data warehouse implementation.
by David R. Schiller
F
orward-thinking companies realize the
importance of an enterprise data warehouse (EDW). They know it is required for
obtaining deep operational and strategic
insights into their businesses. But organizations differ on how difficult they think it is
to implement a data warehouse.
Some are reluctant to begin the project
because of its scope. Others are convinced
they can handle the job internally, unaware
that developing a data warehouse is different from working on any other type of
technology project in its complexity and
level of involvement.
Most companies realize they require the
services and industry-leading expertise
of an accomplished source to guide them
through each step of the process.
Overcome roadblocks
Left alone, many data warehouse implementations run the risk of failure because
of obstacles like competing projects, bad
hardware architecture and improper
design, or a scope too large to achieve
timely results. Data quality issues and other
broken operational processes can also
impede progress.
Furthermore, many organizations lack
corporate vision and support. Executives
who don’t fully appreciate that their data
is an important corporate asset can delay
funding and hinder governance policies.
To overcome these adversities and
ensure the successful completion of a data
warehouse project, best practices suggest
taking advantage of the assistance offered
by a knowledgeable and experienced
group of service providers. The Teradata
Professional Services team, with its group
of consultants committed to the success of
their customers, can work with your company and help manage your data warehouse project. The team has implemented
Photography by iStockphoto
PAGE 1 | Teradata Magazine | September 2009 | ©2009 Teradata Corporation | AR-5964
data warehouse projects worldwide in
multiple industries. From the knowledge
and experience gained through these
engagements, the team has developed a
proven methodology that is adaptable to
any organization.
Plan development
Before any project is started, its scope
must be determined, a plan developed,
and the necessary tools and work force
established. Your Teradata Professional
Services partner will work with you
and your team to create a project worksheet detailing each step and task of the
implementation process. The worksheet
takes into account the expertise your
organization has on its enterprise data,
business rules and success goals; outside
assistance from other third-party vendors;
and the overall product and implementation knowledge the Teradata Professional
Services team can provide. In
the end, you’re given complete
flexibility on whether the
task ownership and resource
staffing are provided by your
company, Teradata or other
third-party organizations. (See
figure, page 2.)
When evaluating the scope
of the project, you and your
team leaders will consult with
your Teradata Professional
Services partner and the
Teradata solution architect
about whether the project
needs to be broken down into
more manageable segments.
The desired outcome for each
90- to 120-day iteration will
services
help determine the tasks, resource requirements, recommended skill sets and assignment duration.
Accommodations are provided in the
plan for project ramp-up and ramp-down;
closeout, internal and external costs; and
scheduling considerations such as vacations
and holidays. A section enables you to track
assumptions until they are agreed upon or
otherwise resolved.
From this worksheet, statements of
work and project plans are created. Not all
personnel will start at once, as some tasks
aren’t undertaken until later. For example,
logical data modeling occurs before
physical data modeling, which is needed
to complete the design of the extract,
transform and load (ETL) processes, and
so on. Some activity overlap will occur, but
the worksheet can be used to generate the
appropriate staffing schedule.
Keep in mind that before any external
team members are brought on board, you
will need adequate lead time to obtain for
them the necessary approvals for security
accounts, access to the Internet and
appropriate facilities, and to establish
physical workspaces.
Figure
Project on-boarding
The Teradata Professional Services partner
and the project manager will collaborate on
preparation details in a phase called project
on-boarding. These activities include
creating a document of the complete team
roster, a project overview and plan, and the
current architecture and tools used. The
project manager sends the document to the
team in advance. Preparing and updating
the team with background information
and task lists before the project starts is
invaluable. It keeps the project on schedule
and the team members informed without
spending client time at the onset.
Team meetings are then held so each function can be explained in detail and information on customer requirements, expectations
and protocol can be distributed and discussed. Through this ramp-up process, team
members gain an understanding of your
company and the project goals and objectives.
Throughout the project’s life cycle, circumstances will arise that could expand the
scope of the project. When this happens,
the Teradata Professional Services partner
or program manager will identify the new
requirements or additional data sources.
This information is used to understand
the impact to the current schedule or
cost. After these are addressed with your
management team, a formal change control
process is created to modify the statement
of work, the project plan and the resources
that are engaged, if necessary.
Step by step
Regarding each member of the task force
as an essential component of the team is
conducive to a successful project, whether the
person is from your organization, Teradata
or another third-party vendor. The table on
page 3 shows a breakdown of the typical team
members from each of these organizations.
An example of a 15-week project, which
provides for two weeks of planning, is shown
below. The chart shows how each team member is involved at specific times depending on
the task and area of expertise. For instance,
the Teradata representative sets up and configures the database the week before the project
officially begins. This continues on through
the first “actual” week. The ETL is designed in
week three by a Teradata team member, but
your in-house and third-party ETL developers don’t get involved until four weeks later.
Enterprise data warehouse phase 1 project detail
Weeks
Project 1, phase 1
Install hardware, software
Database configuration, setup, training customer resources
Data discovery/data profiling
Project/program management
Logical data modeling
Physical data modeling
ETL design
ETL development
BI reporting design
BI reporting development
System/user acceptance testing
Go-live
Teradata resources
Customer resources
Third-party resources
-1
0
3/24 3/31
3/24 3/31
3/24 3/31
1
4/7
4/7
4/7
2
3
4
4/14 4/21 4/28
4/14 4/21 4/28
4/14 4/21 4/28
5
5/5
5/5
5/5
6
7
8
5/12 5/19 5/26
5/12 5/19 5/26
5/12 5/19 5/26
9
10
6/2
6/2
6/2
6/9
6/9
6/9
11
12
13
6/16 6/23 6/30
6/16 6/23 6/30
6/16 6/23 6/30
A 15-week phase 1 enterprise data warehouse design and development project typically follows these steps, divided among Teradata, customer and thirdparty resources. Weeks -1 and 0 are pre-engagement activities.
PAGE 2 | Teradata Magazine | September 2009 | ©2009 Teradata Corporation | AR-5964
With the guidance provided by a flowchart
or workstream from the Teradata Solutions
Methodology, all parties work together
through the different phases of each step in
the project. Identify, analyze, design, develop,
deploy and improve are the phases, which
drill down on each workstream for specific
activities, collateral, templates and tools that
are necessary to achieve each step’s objective.
For example, for the data integration phase
of an EDW project, data gathering and cleansing requirements are developed before the
data loading process can be created and performed. All data standards must be established
in order to build a physical data model. Tasks
required to build a high-level data integration
architecture are included. And, of course, testing is essential to considering this phase of the
data warehouse project ready for production.
Teradata Solutions
Methodology:
Value at every step
M
any IT professionals plan a
data warehouse based on their
experiences with other IT projects. But
building a data warehouse is much more
extensive and intricate, and it needs
its own development strategy. A good
methodology helps ensure a project’s
quality, consistency and timeliness.
The patent-pending Teradata
Solutions Methodology is a collection
of integrated processes, customized
tools and quantifiable metrics—from
initial strategic planning to technical
implementation—that brings value
to every piece of the design and
implementation. User training and
ongoing support are provided to ensure
a smooth transition and expert backing.
Developed out of more than 23
years of experience implementing data
warehouses, the Teradata Solutions
Methodology is continually refined with
best practices learned from the world’s
most successful data warehouse projects.
—D.R.S.
table
Project team resources
in house
teradata
other third party
> Project manager
> DBA
> Developer
> Data architect and technical
leader
> Teradata Professional Services
engagement partner
> Solution architect
> Project manager
> Logical and physical data modeler
> Data architect and technical
leader
> BI report designer
> ETL designer
> Data mappers
> ETL developers
> BI developers
Knowledge transfer
After a data warehouse is installed, there
are always new sources of data to integrate,
additional business questions to ask and
changes to regulatory requirements that
organizations need to quickly address. Your
data warehouse should enable your company to successfully use that data, answer
those questions and breathe easier about
meeting regulations.
To fully utilize the tools and applications
available, you must be knowledgeable about
your data warehouse environment and confident in your ability to manage the system.
For that reason, a driving point of the
Teradata Professional Services philosophy is
to make its customers more self-sufficient
after the implementation is complete but
before its team leaves the customer’s site.
Part of any Teradata Professional
Services engagement includes a variety of
mechanisms to transfer knowledge, such as:
> Client mentoring
> Lunch-and-learn sessions
> Informal and formal training curricula
> Technical forums
Additionally, for two weeks after the
project has gone live, a senior-level Teradata
consultant remains to ensure continuity of
processes, performance and expectations.
Teradata also offers a full suite of managed
services that can be tailored to your company’s needs, providing another option for
ongoing support and access to the entire
community of Teradata experts.
PAGE 3 | Teradata Magazine | September 2009 | ©2009 Teradata Corporation | AR-5964
Ready for business
Choosing the right consultant is key to
the success of your data warehousing
project. A professional services team with
the right skills, knowledge, methodology
and best practices can save many hours of
unnecessary delay and rework. Teradata
consultants average 12 years of experience
in data warehousing, and many are highly
sought technical and thought leaders in
their fields.
It’s undeniable that greater business
value and insight are gained when your
data is transformed into an asset. Working
with Teradata Professional Services, your
organization can expect a high EDW
implementation success rate, and your IT
staff will become more self-sufficient, confident and knowledgeable about the data
warehouse system and what it can provide
your company. T
David R. Schiller, CCP, is a certified
Teradata Professional with more than 27
years of various IT experience as a practitioner, consultant and executive. He has
been with Teradata since 1997 and currently
manages Teradata Professional Services
marketing programs.
online
For more information on Teradata
Professional Services, visit Teradata.com.
insider’s warehouse
TECH2TECH
From the ground up
Business Intelligence Roadmap offers project life cycle methodology
for decision-support applications. by Larissa Moss
A
consultant in the enterprise architecture industry once compared building
systems to building airplanes. While a paper
or model airplane can be constructed with
little forethought, a jet airplane cannot.
Similarly, a small stand-alone application
can be built without careful planning, but
a data warehouse with its many dependent
business intelligence (BI) applications cannot. Yet many project teams try to develop
their data warehouse and BI applications
using no methodology at all. As a result,
they quickly become overwhelmed with
complexity, unanticipated problems, added
tasks, too much rework and wasted time,
and a project that seems out of control.
6 stages with 16 steps
A methodology I co-developed, the Business
Intelligence Roadmap, is designed for data
warehouse and BI applications. The 16 development steps of the roadmap are grouped
into the following engineering stages of justification, planning, analysis, design, construction and deployment. (See table, page 2.)
may be for not implementing a data warehouse or BI solution at your organization.
Interview people at all levels in the organization in order to find out their understanding and expectations of a data warehouse
and BI project. This will dictate whether an
enterprise solution should be architected or
whether building departmental or standalone BI applications is more appropriate.
Discussing the initiative with others will
also help decide the executive sponsor for the
project and whether collective sponsorship
from all business executives will be achieved
for one cohesive enterprise solution.
Planning
Strategic and tactical plans lay out how the
initiative or project will be accomplished.
2 E
nterprise infrastructure
evaluation
Equally important to supporting an evolving data warehouse are:
> Technical infrastructure
Justification
An assessment, which includes cost justification, is made of a business problem
or opportunity. This gives rise to a data
warehouse initiative or BI project.
Photography by iStockphoto
1 Business case assessment
First, define the business drivers and propose
a data warehouse or BI solution. Each BI
application should clearly define the benefits
of either solving a business problem or taking
advantage of an opportunity. You may also
want to find out what types of BI initiatives
your competitors have and what the risks
PAGE 1 | Teradata Magazine | June 2009 | ©2009 Teradata Corporation | AR-5923
• Hardware, software, middleware
• Database management systems,
operating systems
• Network components
• Developer tools, end-user BI tools
> Non-technical infrastructure
• Metadata standards, data naming
standards, enterprise data model
• Data warehouse methodology,
guidelines
• Testing procedures, change control
process, issues management procedures, dispute resolution procedures
Identify the technical and non-technical
infrastructure components you already
have in place and which ones you still need.
Evaluate the completeness and effectiveness
of those components and identify activities
for enhancing or upgrading them.
3 Project planning
The difference between data warehouse/BI
projects and traditional IT projects is the
emphasis on data as opposed to processes.
TECH2TECH
table
insider’s warehouse
and data requirements
for a solution.
Steps for data warehouse/
BI development tracks
Engineering stages
and data warehouse
development steps
Justification
1. B
usiness case
assessment
Planning
2. E
nterprise
infrastructure
evaluation
3. Project planning
Analysis
4. R
equirements
definition
5. Data analysis
6. Prototyping
7. M
etadata repository
analysis
DESIGN
8. Database design
9. ETL design
10. Metadata repository
design
Construction
11. ETL development
12. Application
development
13. Data mining
14. Metadata repository
development
Deployment
15. Implementation
16. Release evaluation
Extract,
transform
and load
(ETL) track
Business
intelligence
(BI) track
Metadata
repository
track
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
√
Before committing to a project deadline,
know the condition of the source files and
the effort it will take to sanitize the data. Be
sure the project plan considers that datarelated activities can take four to five times
the effort of process-related activities. These
activities consist of data modeling, profiling,
standardization and integration, and data
cleansing. Most project deadlines are missed
because the project team underestimated the
amount of dirty data and its impact on the
extract, transform and load (ETL) process.
Analysis
Detailed analysis of the problem or opportunity is performed to identify business
√
√
√
√
√
4 Requirements
definition
Stable requirements
rarely exist on data
warehouse and BI
projects because users
often demand too much
data in an impractical time frame—they
want everything, and
they want it yesterday.
Break the requirements into multiple
packages that you can
deploy in iterations.
Start with requirements
that have the highest
business value, are the
most stable (least likely
to change), and are
sourced from relatively
clean and easily integrated data. Never try to
freeze requirements, but
allow users to change
them when they see BI
capabilities unfold.
5 Data analysis
An in-depth understanding of the data
requirements is produced in this important
development step, including standardized
data names, definitions, domains, business
rules and data quality rules.
The two major activities involved
are top-down data modeling and
bottom-up source data analysis. Topdown data modeling is the creation of a
business data model with the participation of the business users. Bottom-up
source data analysis refers to the data
archaeology performed on the source
data to confirm or refute the business
rules identified during the business data
modeling process.
PAGE 2 | Teradata Magazine | June 2009 | ©2009 Teradata Corporation | AR-5923
Only trained data analysts or data
administrators should perform this step,
not developers.
6 Prototyping
Performing system analysis, design, coding and testing allows users to assess and
change their requirements with less impact
early in the project schedule.
The costs of experimenting with different
database designs, visualization methods,
development tools and programming languages are much lower during prototyping
than during development. Some prototypes
are nothing more than smoke and mirrors
and are used to obtain user buy-in, validate
functional requirements or prove a concept.
Others with longer life spans produce partially functioning pieces of the application.
7 Metadata repository analysis
The four types of metadata and where they
are captured are:
> Business—in a data modeling tool
> Technical—in the relational database
management system (RDBMS) and
online analytical processing (OLAP)
cubes
> Process—in ETL, OLAP and other
developer tools
> Usage—in the RDBMS and manually by the DBA monitoring the data
warehouse environment
These different types of metadata must
be mapped to one another and stored in
a repository. Metadata repositories can be
purchased or built. In either case, document
in a meta model the type of metadata to
capture, how to map it and where to store it.
Design
A product is conceived that will help solve
the business problem or enable the business opportunity.
8 Database design
A database can be either relational or
multi-dimensional, depending on the
underlying RDBMS, the purpose of the
database, and its access paths and BI tools.
A relational design offers maximum flexibility for an operational data store (ODS)
or an enterprise data warehouse (EDW). A
multi-dimensional design offers maximum
performance and is more common for
data marts that are tailored toward specific
reporting patterns or BI applications.
The project’s specific performance considerations should influence the database
schema you choose as well as the physical
components, such as indexing strategy, data
set placement, partitioning and clustering.
9 ETL design
ETL functions are designed for different
processes:
> Initial population is very similar to a system conversion. The main task is to map
the selected data elements from the current source files to the target columns in
the data warehouse databases.
> Historical population is a process that
loads historical data from old files.
This set of specifications will be slightly
different because historical data has
usually been archived to off-line storage
devices. Since the record layouts of the
files and databases change over time,
the programs for the historical population have to recognize those changes.
> Incremental update refers to the ongoing
updates (i.e., inserts) to the data warehouse databases. Two things must be
considered for the incremental update:
whether to extract all records from the
source files or only the changed records
(deltas), and whether to pull data from
the source files or push out the data
from the source systems. In many data
warehouse environments, the optimum
choice is to have push-out deltas only.
10 Metadata repository design
If built in-house, several metadata repository
structures must be designed: the database, the
migration programs that load and link the
Figure
Workflow for data warehouse/
BI development tracks
A 16-step methodology for designing data warehouse and business intelligence (BI) applications is grouped
into the engineering stages of justification, planning, analysis, design, construction and deployment.
metadata from the tools where it is captured,
the interface programs that communicate
with the tools and users, and the online help
function. If purchased, the metadata repository must be installed and tested.
Construction
The conceived product is built, and the
design elements are incorporated.
11 ETL development
Implementing the ETL process is almost
always done through an ETL tool. Some
ETL tools are more sophisticated than
others. Thus, in some cases, additional code
must be written to supplement their functions. Because the ETL process is the most
complicated process in the data warehouse
environment, it must be thoroughly and
PAGE 3 | Teradata Magazine | June 2009 | ©2009 Teradata Corporation | AR-5923
formally tested. This is done with a formal
test plan, which contains test cases, test
sequence, expected test results, actual test
results and a test log. Recommended types
of tests are unit, end-to-end integration
and regression, performance or stress,
quality assurance (pre-production), and
user acceptance.
12 Application development
The various prototyping results are used
to construct a production-worthy frontend BI application. This can be a state-ofthe-practice OLAP cube with reports and
canned or parameterized queries. It could
also be a state-of-the-art dashboard or
scorecard. In either case, the BI application
is subjected to the same rigorous testing
activities as the ETL process.
13 Data mining
A specialized data mining application uses
a tool to execute data mining operations
against a data pool. This pool is based on
an analytical data model, which is usually
developed by a statistician.
Data mining activities include identifying the business problem domain; collecting, consolidating, cleansing and preparing the data for the analytical data model;
and populating the database. The results
are often displayed visually in charts (bar,
pie and lift), decision trees, scatter plots
and histograms.
14 M
etadata repository
development
Creating a database, writing metadataspecific migration programs to extract the
metadata, coding interface programs to
deliver metadata to the users, and providing an online help function are some activities to developing a metadata repository.
Another development step is writing
ETL-specific metadata programs. These
programs capture load statistics, reconciliation totals, data quality statistics and error
statistics produced during the ETL runs.
After each run, these statistics are loaded
into the metadata repository.
Deployment
The finished product is implemented and
its effectiveness measured to determine
whether the solution meets, exceeds or fails
end-user expectations. Additionally, a predefined time frame has been established to
determine return on investment (ROI).
15 Implementation
After the programs are tested, the data
warehouse, BI application and metadata
repository are ready for production. The
databases and program libraries that will
house the ETL and BI programs must be
created. Once the data has been loaded
into the databases, all security measures
should be tested again. The users are then
trained, and the support functions—such
as a help desk, maintenance of the data
warehouse databases, scheduling and
running ETL jobs, performance monitoring and tuning, and monitoring usage—
are initiated.
16 Release evaluation
A data warehouse is not a system but
an evolving and expanding environment.
It is beneficial, therefore, to review lessons learned at the end of every project.
Not only does this enable the team to
anticipate and prevent problems in future
projects but it also helps to streamline the
development process.
Any tools, techniques, guidelines and
processes that were not useful should be reevaluated and adjusted or, possibly, discarded.
Any missed deadlines, cost overruns, disputes
and resolutions should be examined, and
adjustments to the processes should be made
before the next project begins.
The release evaluation review is also the
perfect forum for repackaging dropped
data or functionality and reprioritizing the
next projects.
Data warehouse/
BI development tracks
Although the development steps are presented sequentially and mapped into the
engineering stages, these types of projects
are rarely executed in this order. Some
steps could be combined or bypassed. At
a minimum, most projects will divide the
development effort into multiple tracks.
(See figure, page 3.)
The two most common tracks are ETL
and BI application. A third track may also
be needed: metadata repository. If data
mining is a full-time activity at your company, the data mining step can be separated
into its own track. The table on page 2
shows which steps apply to each track.
While each track can have its own
team and set of activities, they are
interdependent. All tracks start and end
PAGE 4 | Teradata Magazine | June 2009 | ©2009 Teradata Corporation | AR-5923
their activities at the same time, but the
activities during the analysis, design and
construction stages may diverge in terms
of duration and work organization.
The back-end ETL track may spend
more time in analysis before designing and
developing the ETL process, and the frontend BI application track may prototype
extensively. The metadata repository track
may be organized as a separate but parallel
development project, or it may simply
involve buying and installing a metadata
repository product. Since discoveries made
on one track often affect the others, it is
important to manage all resources as one
project team.
Supportive methodology
Successful data warehouse and BI initiatives
evolve into complicated enterprise-wide
decision-support environments. A strong
foundation is required to support various
BI applications for different types of users.
To build such a foundation, the question is
not whether a methodology is needed, but
what type is the best option.
The data warehouse is built in increments to accommodate requirements that
are inevitably evolving and morphing.
Its methodology, therefore, must support
periodic, spiral architectural redesigns and
have a strong focus on data and powerful
enterprise infrastructure activities.
Data warehouse projects are full of challenges, but having to remember hundreds
of tasks should not be one of them. That’s
why a good methodology specifically
designed for data warehouse and BI projects is necessary.
Planning of this sort will help strengthen
any company’s BI investment. T
Larissa Moss, president of Method Focus Inc.,
has 29 years of IT experience with a focus
on data warehousing and data management. She is a worldwide speaker and widely
published author. Larissa can be reached at
methodfocus@earthlink.net.
Purchase answer to see full
attachment