SlideShare una empresa de Scribd logo
1 de 4
Descargar para leer sin conexión
Something about DataStage, DataStage Administration, Job Designing,Developing, DataStage troubleshooting, DataStage Installation & Configuration, ETL, DataWareHousing, DB2,
Teradata, Oracle and Scripting.
Nuts & Bolts of DataStage
Home Interview Questions DataStage Scenarios Series Posts E­Books About Me !!
Monday, July 07, 2014
DataStage Scenario Problem ­­>  DataStage Scenario ­ Problem9
Solution Design :
a) Job Design :   
Below design will achieve the output as per our requirement.
b) Transformer Stage Properties : 
 Input Source ­ a Seq file
Ouput target ­ 3 seq file with data
Map the input column to all 3 outputs.
DataStage Scenario ­ Design9 ­ job1
Total Pageviews
1 4 5 4 6 2 0
Search
Try Me
DataSet in DataStage
Issuing commands to a Queue Manager (runmqsc)
Hash Files in DataStage
XMeta DB : Datastage Repository
InfoSphere DataStage Jobstatus returned Codes from
dsjob
Conductor Node in Datastage
Schema File in Datastage
Sort stage to remove duplicate
14 Good design tips in Datastage
Datastage Coding Checklist
Must Reads
2   More    Next Blog» Create Blog   Sign In
Now setup constraints (condition) for each Link.
File A ­  Mod(DSLink2.col1,9)=1 or Mod(DSLink2.col1,9)=2 or Mod(DSLink2.col1,9)=3
File B ­  Mod(DSLink2.col1,9)=4 or Mod(DSLink2.col1,9)=4 or Mod(DSLink2.col1,9)=6
File C ­  Mod(DSLink2.col1,9)=7 or Mod(DSLink2.col1,9)=8 or Mod(DSLink2.col1,9)=0
and Now..Compile and Run the job :­)
Like the Facebook Page & join Group
https://www.facebook.com/DataStage4you
https://www.facebook.com/groups/DataStage4you
https://twitter.com/datastage4you
For WHATSAPP group , drop a msg to 91­88­00­906098 
By Atul Singh at 11:08  0 Comments
Labels: constrain, DataStage, design, Designer, develop, Job, Seq File, transformer
Get daily dose of Tech Food
Email address... Submit
DataStage4You
111 have us in circles View all
Follow
tech foodies
▼  2014 (103)
►  October (7)
►  September (9)
►  August (5)
▼  July (12)
Oracle SQL Tuning Tips ­ 2
Oracle SQL Tuning Tips ­ 1
DataStage Custom Routine to Get a File
Size
Connect to Oracle DB with Linux Shell
Script
Navigating the many paths of metadata for
DataStag...
Some DataStage Job design
DataStage Scenario ­ Problem20
Handling Filename with Spaces in Linux
Mongo DB ­ Installation and Configuration
Column Import Stage ­ Read a Fixed width
File
DataStage Scenario ­ Design9 ­ job1
Count Rows and Calculate Sum in same
Aggregator
Blog Archive
Newer Post Older PostHome
Subscribe to: Post Comments (Atom)
0 Comments DataStage4You  Login
Sort by Best Share ⤤
Start the discussion…
Be the first to comment.
Subscribe✉ Add Disqus to your sited Privacy
Favorite ★
►  June (10)
►  May (13)
►  April (10)
►  March (9)
►  February (16)
►  January (12)
►  2013 (167)
►  2012 (175)
►  2011 (8)
Administration 
application  authorities
client  Code  column
commands  Concept
Configuration 
create  Data  database  DataSet
DataStage  DataWareHouse  DB2  DBMS
debug  delete  design  develop
difference  director
Documentation  dsenv  dsjob  DSRPC 
environment Errors  ETL 
file 
function 
Information input  install  Interview 
Job  keys  Link Linux list 
Logging Logical  logs lookup 
managers  message  queue
Metadata Model  MQ  names
Optimizing  Oracle 
output  Parallel  parameter  partition
performance  Physical 
port  problem  process 
Project  Putty Questions 
remove 
routine  rows 
scenario  Schema  Script 
Tags Cloud
&PH& 421  advantage Agents aggregator
Answers  architecture ASB attribute 
backup basic binary block books Buffer certification  change
channel  checkpoint  cleanup  clear 
Column  Generator  compiler 
Conceptual  conductor  container  copy
counter  Crontab 
deadlock  deploy 
dimension  Dimensional 
DSparam  dump
duplicate encrypt engine  exception
execution export fact factless  FAQ  FileSet filter free ftp
fun  fundamentals granularity  Guest hadoop handling
hash  head hide horizontal  Host huge  hyperlink  import  increase
index  issue  istool Java
jdbc  join  leaders  listener load
local locks  Login  macro mail
maintenance  memory  merge 
modify Monitor  MQSC multiple 
NLS  node  notes  notification  odbc  odbc.ini  operator
orchadmin  ORLogging  orphan  OS  osh
package  Parallelism 
password peek  Perl phantom  pivot
player  Practices  profile
programming  purge  read registry
reject  release  report  Resource  Restart  Roles
row  generator  RTLogging  run  sample  SCD
scheduler  score  Scratch  section
Seq  File  sequence  Server  Service  Setting
Shell  shell  scripting 
sort source  SQL  stages
Start  Stop 
surrogate  table  target  teradata
tips  tool  transformer 
Troubleshoot  Tutorial  Unix User
Utility  UV  variables 
warnings  WAS  websphere
windows  XMETA 
session 
Share  shortcuts  show  slowly
snowflake solution  space  SSH 
Standards  Star  statistics  status  storage
switch system  tail  temporary 
time  trace  transformation  trigger
tuning  type unique 
uvodbc.config  version  videos  view
Vincent  McBurney  Virtual 
write Write Range Map  xml z/OS
The postings on this site are my own and don't necessarily represent IBM's or other companies positions, strategies or opinions. All content provided on this blog is for informational purposes only. The owner of this
blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. The owner will not be liable for any errors or omissions in this information
nor for the availability of this information. The owner will not be liable for any losses, injuries, or damages from the display or use of his information. //­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­
Disclaimer
Did you find this Blog helpful ?? Let me know @ www.facebook.com/datastage4you. Ethereal template. Powered by Blogger.

Más contenido relacionado

La actualidad más candente

Oracle JSON treatment evolution - from 12.1 to 18 AOUG-2018
Oracle JSON treatment evolution - from 12.1 to 18 AOUG-2018Oracle JSON treatment evolution - from 12.1 to 18 AOUG-2018
Oracle JSON treatment evolution - from 12.1 to 18 AOUG-2018Alexander Tokarev
 
Low Level CPU Performance Profiling Examples
Low Level CPU Performance Profiling ExamplesLow Level CPU Performance Profiling Examples
Low Level CPU Performance Profiling ExamplesTanel Poder
 
An Introduction to Netezza
An Introduction to NetezzaAn Introduction to Netezza
An Introduction to NetezzaVijaya Chandrika
 
Bucketing 2.0: Improve Spark SQL Performance by Removing Shuffle
Bucketing 2.0: Improve Spark SQL Performance by Removing ShuffleBucketing 2.0: Improve Spark SQL Performance by Removing Shuffle
Bucketing 2.0: Improve Spark SQL Performance by Removing ShuffleDatabricks
 
Power JSON with PostgreSQL
Power JSON with PostgreSQLPower JSON with PostgreSQL
Power JSON with PostgreSQLEDB
 
C* Keys: Partitioning, Clustering, & CrossFit (Adam Hutson, DataScale) | Cass...
C* Keys: Partitioning, Clustering, & CrossFit (Adam Hutson, DataScale) | Cass...C* Keys: Partitioning, Clustering, & CrossFit (Adam Hutson, DataScale) | Cass...
C* Keys: Partitioning, Clustering, & CrossFit (Adam Hutson, DataScale) | Cass...DataStax
 
Big data 101 for beginners riga dev days
Big data 101 for beginners riga dev daysBig data 101 for beginners riga dev days
Big data 101 for beginners riga dev daysDuyhai Doan
 
How to teach an elephant to rock'n'roll
How to teach an elephant to rock'n'rollHow to teach an elephant to rock'n'roll
How to teach an elephant to rock'n'rollPGConf APAC
 
Delta Lake: Optimizing Merge
Delta Lake: Optimizing MergeDelta Lake: Optimizing Merge
Delta Lake: Optimizing MergeDatabricks
 
Materialized Column: An Efficient Way to Optimize Queries on Nested Columns
Materialized Column: An Efficient Way to Optimize Queries on Nested ColumnsMaterialized Column: An Efficient Way to Optimize Queries on Nested Columns
Materialized Column: An Efficient Way to Optimize Queries on Nested ColumnsDatabricks
 
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...InfluxData
 
From Postgres to Cassandra (Rimas Silkaitis, Heroku) | C* Summit 2016
From Postgres to Cassandra (Rimas Silkaitis, Heroku) | C* Summit 2016From Postgres to Cassandra (Rimas Silkaitis, Heroku) | C* Summit 2016
From Postgres to Cassandra (Rimas Silkaitis, Heroku) | C* Summit 2016DataStax
 
Relational databases for BigData
Relational databases for BigDataRelational databases for BigData
Relational databases for BigDataAlexander Tokarev
 
Accessing Databases from R
Accessing Databases from RAccessing Databases from R
Accessing Databases from RJeffrey Breen
 
Introduction to Google BigQuery
Introduction to Google BigQueryIntroduction to Google BigQuery
Introduction to Google BigQueryCsaba Toth
 
Five Database Mistakes and how to fix them -- Confoo Vancouver
Five Database Mistakes and how to fix them -- Confoo VancouverFive Database Mistakes and how to fix them -- Confoo Vancouver
Five Database Mistakes and how to fix them -- Confoo VancouverDave Stokes
 
Sasi, cassandra on full text search ride
Sasi, cassandra on full text search rideSasi, cassandra on full text search ride
Sasi, cassandra on full text search rideDuyhai Doan
 
MongoDB Aggregation Performance
MongoDB Aggregation PerformanceMongoDB Aggregation Performance
MongoDB Aggregation PerformanceMongoDB
 
Advanced SQL For Data Scientists
Advanced SQL For Data ScientistsAdvanced SQL For Data Scientists
Advanced SQL For Data ScientistsDatabricks
 

La actualidad más candente (20)

Oracle JSON treatment evolution - from 12.1 to 18 AOUG-2018
Oracle JSON treatment evolution - from 12.1 to 18 AOUG-2018Oracle JSON treatment evolution - from 12.1 to 18 AOUG-2018
Oracle JSON treatment evolution - from 12.1 to 18 AOUG-2018
 
Low Level CPU Performance Profiling Examples
Low Level CPU Performance Profiling ExamplesLow Level CPU Performance Profiling Examples
Low Level CPU Performance Profiling Examples
 
Cloud DWH deep dive
Cloud DWH deep diveCloud DWH deep dive
Cloud DWH deep dive
 
An Introduction to Netezza
An Introduction to NetezzaAn Introduction to Netezza
An Introduction to Netezza
 
Bucketing 2.0: Improve Spark SQL Performance by Removing Shuffle
Bucketing 2.0: Improve Spark SQL Performance by Removing ShuffleBucketing 2.0: Improve Spark SQL Performance by Removing Shuffle
Bucketing 2.0: Improve Spark SQL Performance by Removing Shuffle
 
Power JSON with PostgreSQL
Power JSON with PostgreSQLPower JSON with PostgreSQL
Power JSON with PostgreSQL
 
C* Keys: Partitioning, Clustering, & CrossFit (Adam Hutson, DataScale) | Cass...
C* Keys: Partitioning, Clustering, & CrossFit (Adam Hutson, DataScale) | Cass...C* Keys: Partitioning, Clustering, & CrossFit (Adam Hutson, DataScale) | Cass...
C* Keys: Partitioning, Clustering, & CrossFit (Adam Hutson, DataScale) | Cass...
 
Big data 101 for beginners riga dev days
Big data 101 for beginners riga dev daysBig data 101 for beginners riga dev days
Big data 101 for beginners riga dev days
 
How to teach an elephant to rock'n'roll
How to teach an elephant to rock'n'rollHow to teach an elephant to rock'n'roll
How to teach an elephant to rock'n'roll
 
Delta Lake: Optimizing Merge
Delta Lake: Optimizing MergeDelta Lake: Optimizing Merge
Delta Lake: Optimizing Merge
 
Materialized Column: An Efficient Way to Optimize Queries on Nested Columns
Materialized Column: An Efficient Way to Optimize Queries on Nested ColumnsMaterialized Column: An Efficient Way to Optimize Queries on Nested Columns
Materialized Column: An Efficient Way to Optimize Queries on Nested Columns
 
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...
InfluxDB IOx Tech Talks: Query Engine Design and the Rust-Based DataFusion in...
 
From Postgres to Cassandra (Rimas Silkaitis, Heroku) | C* Summit 2016
From Postgres to Cassandra (Rimas Silkaitis, Heroku) | C* Summit 2016From Postgres to Cassandra (Rimas Silkaitis, Heroku) | C* Summit 2016
From Postgres to Cassandra (Rimas Silkaitis, Heroku) | C* Summit 2016
 
Relational databases for BigData
Relational databases for BigDataRelational databases for BigData
Relational databases for BigData
 
Accessing Databases from R
Accessing Databases from RAccessing Databases from R
Accessing Databases from R
 
Introduction to Google BigQuery
Introduction to Google BigQueryIntroduction to Google BigQuery
Introduction to Google BigQuery
 
Five Database Mistakes and how to fix them -- Confoo Vancouver
Five Database Mistakes and how to fix them -- Confoo VancouverFive Database Mistakes and how to fix them -- Confoo Vancouver
Five Database Mistakes and how to fix them -- Confoo Vancouver
 
Sasi, cassandra on full text search ride
Sasi, cassandra on full text search rideSasi, cassandra on full text search ride
Sasi, cassandra on full text search ride
 
MongoDB Aggregation Performance
MongoDB Aggregation PerformanceMongoDB Aggregation Performance
MongoDB Aggregation Performance
 
Advanced SQL For Data Scientists
Advanced SQL For Data ScientistsAdvanced SQL For Data Scientists
Advanced SQL For Data Scientists
 

Destacado

Data stage scenario design2 - job3
Data stage scenario   design2 - job3Data stage scenario   design2 - job3
Data stage scenario design2 - job3Naresh Bala
 
Datastage 8.1 real time q
Datastage 8.1 real time qDatastage 8.1 real time q
Datastage 8.1 real time qNaresh Bala
 
Data stage scenario design 2 - job1
Data stage scenario   design 2 - job1Data stage scenario   design 2 - job1
Data stage scenario design 2 - job1Naresh Bala
 
Data stage faqs datastage faqs
Data stage faqs  datastage faqsData stage faqs  datastage faqs
Data stage faqs datastage faqsNaresh Bala
 
Oracle PL/SQL online training | PL/SQL online Training
Oracle PL/SQL online training | PL/SQL online TrainingOracle PL/SQL online training | PL/SQL online Training
Oracle PL/SQL online training | PL/SQL online Trainingsuresh
 
Sql interview question part 4
Sql interview question part 4Sql interview question part 4
Sql interview question part 4kaashiv1
 
Unix interview questions
Unix interview questionsUnix interview questions
Unix interview questionsKalyan Hadoop
 
Sql interview question part 2
Sql interview question part 2Sql interview question part 2
Sql interview question part 2kaashiv1
 
Data stage scenario design6 - job1
Data stage scenario   design6 - job1Data stage scenario   design6 - job1
Data stage scenario design6 - job1Naresh Bala
 
Shell Scripting With Arguments
Shell Scripting With ArgumentsShell Scripting With Arguments
Shell Scripting With ArgumentsTechronology Inc.
 
Data stage interview questions and answers|DataStage FAQS
Data stage interview questions and answers|DataStage FAQSData stage interview questions and answers|DataStage FAQS
Data stage interview questions and answers|DataStage FAQSBigClasses.com
 
Datastage free tutorial
Datastage free tutorialDatastage free tutorial
Datastage free tutorialtekslate1
 
Datastage real time scenario
Datastage real time scenarioDatastage real time scenario
Datastage real time scenarioNaresh Bala
 
data stage-material
data stage-materialdata stage-material
data stage-materialRajesh Kv
 

Destacado (17)

Data stage scenario design2 - job3
Data stage scenario   design2 - job3Data stage scenario   design2 - job3
Data stage scenario design2 - job3
 
Datastage 8.1 real time q
Datastage 8.1 real time qDatastage 8.1 real time q
Datastage 8.1 real time q
 
Data stage scenario design 2 - job1
Data stage scenario   design 2 - job1Data stage scenario   design 2 - job1
Data stage scenario design 2 - job1
 
Data stage faqs datastage faqs
Data stage faqs  datastage faqsData stage faqs  datastage faqs
Data stage faqs datastage faqs
 
Oracle PL/SQL online training | PL/SQL online Training
Oracle PL/SQL online training | PL/SQL online TrainingOracle PL/SQL online training | PL/SQL online Training
Oracle PL/SQL online training | PL/SQL online Training
 
Sql interview question part 4
Sql interview question part 4Sql interview question part 4
Sql interview question part 4
 
Unix interview questions
Unix interview questionsUnix interview questions
Unix interview questions
 
Sql interview question part 2
Sql interview question part 2Sql interview question part 2
Sql interview question part 2
 
Data stage scenario design6 - job1
Data stage scenario   design6 - job1Data stage scenario   design6 - job1
Data stage scenario design6 - job1
 
Shell Scripting With Arguments
Shell Scripting With ArgumentsShell Scripting With Arguments
Shell Scripting With Arguments
 
SQL Differences SQL Interview Questions
SQL Differences  SQL Interview QuestionsSQL Differences  SQL Interview Questions
SQL Differences SQL Interview Questions
 
Data stage interview questions and answers|DataStage FAQS
Data stage interview questions and answers|DataStage FAQSData stage interview questions and answers|DataStage FAQS
Data stage interview questions and answers|DataStage FAQS
 
Datastage free tutorial
Datastage free tutorialDatastage free tutorial
Datastage free tutorial
 
Datastage real time scenario
Datastage real time scenarioDatastage real time scenario
Datastage real time scenario
 
Ibm info sphere datastage tutorial part 1 architecture examples
Ibm info sphere datastage tutorial part 1  architecture examplesIbm info sphere datastage tutorial part 1  architecture examples
Ibm info sphere datastage tutorial part 1 architecture examples
 
data stage-material
data stage-materialdata stage-material
data stage-material
 
Intorduction to Datapower
Intorduction to DatapowerIntorduction to Datapower
Intorduction to Datapower
 

Similar a Data stage scenario design9 - job1

Testing database content with DBUnit. My experience.
Testing database content with DBUnit. My experience.Testing database content with DBUnit. My experience.
Testing database content with DBUnit. My experience.Serhii Kartashov
 
New Developments in Spark
New Developments in SparkNew Developments in Spark
New Developments in SparkDatabricks
 
High Performance Jdbc
High Performance JdbcHigh Performance Jdbc
High Performance JdbcSam Pattsin
 
Accessing Databases from R
Accessing Databases from RAccessing Databases from R
Accessing Databases from Rkmettler
 
SQL Server It Just Runs Faster
SQL Server It Just Runs FasterSQL Server It Just Runs Faster
SQL Server It Just Runs FasterBob Ward
 
Jump Start into Apache® Spark™ and Databricks
Jump Start into Apache® Spark™ and DatabricksJump Start into Apache® Spark™ and Databricks
Jump Start into Apache® Spark™ and DatabricksDatabricks
 
Frustration-Reduced PySpark: Data engineering with DataFrames
Frustration-Reduced PySpark: Data engineering with DataFramesFrustration-Reduced PySpark: Data engineering with DataFrames
Frustration-Reduced PySpark: Data engineering with DataFramesIlya Ganelin
 
Best Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache SparkBest Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache SparkDatabricks
 
Sql server engine cpu cache as the new ram
Sql server engine cpu cache as the new ramSql server engine cpu cache as the new ram
Sql server engine cpu cache as the new ramChris Adkin
 
Sql on hadoop the secret presentation.3pptx
Sql on hadoop  the secret presentation.3pptxSql on hadoop  the secret presentation.3pptx
Sql on hadoop the secret presentation.3pptxPaulo Alonso
 
Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu Yong
Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu YongUnlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu Yong
Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu YongCeph Community
 
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Citus Data
 
Successfully migrating existing databases to Azure
Successfully migrating existing databases to AzureSuccessfully migrating existing databases to Azure
Successfully migrating existing databases to AzureRed Gate Software
 
Structuring Apache Spark 2.0: SQL, DataFrames, Datasets And Streaming - by Mi...
Structuring Apache Spark 2.0: SQL, DataFrames, Datasets And Streaming - by Mi...Structuring Apache Spark 2.0: SQL, DataFrames, Datasets And Streaming - by Mi...
Structuring Apache Spark 2.0: SQL, DataFrames, Datasets And Streaming - by Mi...Databricks
 
Structuring Spark: DataFrames, Datasets, and Streaming
Structuring Spark: DataFrames, Datasets, and StreamingStructuring Spark: DataFrames, Datasets, and Streaming
Structuring Spark: DataFrames, Datasets, and StreamingDatabricks
 
Structuring Spark: DataFrames, Datasets, and Streaming by Michael Armbrust
Structuring Spark: DataFrames, Datasets, and Streaming by Michael ArmbrustStructuring Spark: DataFrames, Datasets, and Streaming by Michael Armbrust
Structuring Spark: DataFrames, Datasets, and Streaming by Michael ArmbrustSpark Summit
 
Building a SIMD Supported Vectorized Native Engine for Spark SQL
Building a SIMD Supported Vectorized Native Engine for Spark SQLBuilding a SIMD Supported Vectorized Native Engine for Spark SQL
Building a SIMD Supported Vectorized Native Engine for Spark SQLDatabricks
 
Squeak DBX
Squeak DBXSqueak DBX
Squeak DBXESUG
 

Similar a Data stage scenario design9 - job1 (20)

Testing database content with DBUnit. My experience.
Testing database content with DBUnit. My experience.Testing database content with DBUnit. My experience.
Testing database content with DBUnit. My experience.
 
New Developments in Spark
New Developments in SparkNew Developments in Spark
New Developments in Spark
 
High Performance Jdbc
High Performance JdbcHigh Performance Jdbc
High Performance Jdbc
 
Accessing Databases from R
Accessing Databases from RAccessing Databases from R
Accessing Databases from R
 
SQL Server It Just Runs Faster
SQL Server It Just Runs FasterSQL Server It Just Runs Faster
SQL Server It Just Runs Faster
 
Jump Start into Apache® Spark™ and Databricks
Jump Start into Apache® Spark™ and DatabricksJump Start into Apache® Spark™ and Databricks
Jump Start into Apache® Spark™ and Databricks
 
4 jdbc step1
4 jdbc step14 jdbc step1
4 jdbc step1
 
Frustration-Reduced PySpark: Data engineering with DataFrames
Frustration-Reduced PySpark: Data engineering with DataFramesFrustration-Reduced PySpark: Data engineering with DataFrames
Frustration-Reduced PySpark: Data engineering with DataFrames
 
Best Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache SparkBest Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache Spark
 
Sql server engine cpu cache as the new ram
Sql server engine cpu cache as the new ramSql server engine cpu cache as the new ram
Sql server engine cpu cache as the new ram
 
NoSQL Infrastructure
NoSQL InfrastructureNoSQL Infrastructure
NoSQL Infrastructure
 
Sql on hadoop the secret presentation.3pptx
Sql on hadoop  the secret presentation.3pptxSql on hadoop  the secret presentation.3pptx
Sql on hadoop the secret presentation.3pptx
 
Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu Yong
Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu YongUnlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu Yong
Unlock Bigdata Analytic Efficiency with Ceph Data Lake - Zhang Jian, Fu Yong
 
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
 
Successfully migrating existing databases to Azure
Successfully migrating existing databases to AzureSuccessfully migrating existing databases to Azure
Successfully migrating existing databases to Azure
 
Structuring Apache Spark 2.0: SQL, DataFrames, Datasets And Streaming - by Mi...
Structuring Apache Spark 2.0: SQL, DataFrames, Datasets And Streaming - by Mi...Structuring Apache Spark 2.0: SQL, DataFrames, Datasets And Streaming - by Mi...
Structuring Apache Spark 2.0: SQL, DataFrames, Datasets And Streaming - by Mi...
 
Structuring Spark: DataFrames, Datasets, and Streaming
Structuring Spark: DataFrames, Datasets, and StreamingStructuring Spark: DataFrames, Datasets, and Streaming
Structuring Spark: DataFrames, Datasets, and Streaming
 
Structuring Spark: DataFrames, Datasets, and Streaming by Michael Armbrust
Structuring Spark: DataFrames, Datasets, and Streaming by Michael ArmbrustStructuring Spark: DataFrames, Datasets, and Streaming by Michael Armbrust
Structuring Spark: DataFrames, Datasets, and Streaming by Michael Armbrust
 
Building a SIMD Supported Vectorized Native Engine for Spark SQL
Building a SIMD Supported Vectorized Native Engine for Spark SQLBuilding a SIMD Supported Vectorized Native Engine for Spark SQL
Building a SIMD Supported Vectorized Native Engine for Spark SQL
 
Squeak DBX
Squeak DBXSqueak DBX
Squeak DBX
 

Data stage scenario design9 - job1

  • 1. Something about DataStage, DataStage Administration, Job Designing,Developing, DataStage troubleshooting, DataStage Installation & Configuration, ETL, DataWareHousing, DB2, Teradata, Oracle and Scripting. Nuts & Bolts of DataStage Home Interview Questions DataStage Scenarios Series Posts E­Books About Me !! Monday, July 07, 2014 DataStage Scenario Problem ­­>  DataStage Scenario ­ Problem9 Solution Design : a) Job Design :    Below design will achieve the output as per our requirement. b) Transformer Stage Properties :   Input Source ­ a Seq file Ouput target ­ 3 seq file with data Map the input column to all 3 outputs. DataStage Scenario ­ Design9 ­ job1 Total Pageviews 1 4 5 4 6 2 0 Search Try Me DataSet in DataStage Issuing commands to a Queue Manager (runmqsc) Hash Files in DataStage XMeta DB : Datastage Repository InfoSphere DataStage Jobstatus returned Codes from dsjob Conductor Node in Datastage Schema File in Datastage Sort stage to remove duplicate 14 Good design tips in Datastage Datastage Coding Checklist Must Reads 2   More    Next Blog» Create Blog   Sign In
  • 2. Now setup constraints (condition) for each Link. File A ­  Mod(DSLink2.col1,9)=1 or Mod(DSLink2.col1,9)=2 or Mod(DSLink2.col1,9)=3 File B ­  Mod(DSLink2.col1,9)=4 or Mod(DSLink2.col1,9)=4 or Mod(DSLink2.col1,9)=6 File C ­  Mod(DSLink2.col1,9)=7 or Mod(DSLink2.col1,9)=8 or Mod(DSLink2.col1,9)=0 and Now..Compile and Run the job :­) Like the Facebook Page & join Group https://www.facebook.com/DataStage4you https://www.facebook.com/groups/DataStage4you https://twitter.com/datastage4you For WHATSAPP group , drop a msg to 91­88­00­906098  By Atul Singh at 11:08  0 Comments Labels: constrain, DataStage, design, Designer, develop, Job, Seq File, transformer Get daily dose of Tech Food Email address... Submit DataStage4You 111 have us in circles View all Follow tech foodies ▼  2014 (103) ►  October (7) ►  September (9) ►  August (5) ▼  July (12) Oracle SQL Tuning Tips ­ 2 Oracle SQL Tuning Tips ­ 1 DataStage Custom Routine to Get a File Size Connect to Oracle DB with Linux Shell Script Navigating the many paths of metadata for DataStag... Some DataStage Job design DataStage Scenario ­ Problem20 Handling Filename with Spaces in Linux Mongo DB ­ Installation and Configuration Column Import Stage ­ Read a Fixed width File DataStage Scenario ­ Design9 ­ job1 Count Rows and Calculate Sum in same Aggregator Blog Archive
  • 3. Newer Post Older PostHome Subscribe to: Post Comments (Atom) 0 Comments DataStage4You  Login Sort by Best Share ⤤ Start the discussion… Be the first to comment. Subscribe✉ Add Disqus to your sited Privacy Favorite ★ ►  June (10) ►  May (13) ►  April (10) ►  March (9) ►  February (16) ►  January (12) ►  2013 (167) ►  2012 (175) ►  2011 (8) Administration  application  authorities client  Code  column commands  Concept Configuration  create  Data  database  DataSet DataStage  DataWareHouse  DB2  DBMS debug  delete  design  develop difference  director Documentation  dsenv  dsjob  DSRPC  environment Errors  ETL  file  function  Information input  install  Interview  Job  keys  Link Linux list  Logging Logical  logs lookup  managers  message  queue Metadata Model  MQ  names Optimizing  Oracle  output  Parallel  parameter  partition performance  Physical  port  problem  process  Project  Putty Questions  remove  routine  rows  scenario  Schema  Script  Tags Cloud &PH& 421  advantage Agents aggregator Answers  architecture ASB attribute  backup basic binary block books Buffer certification  change channel  checkpoint  cleanup  clear  Column  Generator  compiler  Conceptual  conductor  container  copy counter  Crontab  deadlock  deploy  dimension  Dimensional  DSparam  dump duplicate encrypt engine  exception execution export fact factless  FAQ  FileSet filter free ftp fun  fundamentals granularity  Guest hadoop handling hash  head hide horizontal  Host huge  hyperlink  import  increase index  issue  istool Java jdbc  join  leaders  listener load local locks  Login  macro mail maintenance  memory  merge  modify Monitor  MQSC multiple  NLS  node  notes  notification  odbc  odbc.ini  operator orchadmin  ORLogging  orphan  OS  osh package  Parallelism  password peek  Perl phantom  pivot player  Practices  profile programming  purge  read registry reject  release  report  Resource  Restart  Roles row  generator  RTLogging  run  sample  SCD scheduler  score  Scratch  section
  • 4. Seq  File  sequence  Server  Service  Setting Shell  shell  scripting  sort source  SQL  stages Start  Stop  surrogate  table  target  teradata tips  tool  transformer  Troubleshoot  Tutorial  Unix User Utility  UV  variables  warnings  WAS  websphere windows  XMETA  session  Share  shortcuts  show  slowly snowflake solution  space  SSH  Standards  Star  statistics  status  storage switch system  tail  temporary  time  trace  transformation  trigger tuning  type unique  uvodbc.config  version  videos  view Vincent  McBurney  Virtual  write Write Range Map  xml z/OS The postings on this site are my own and don't necessarily represent IBM's or other companies positions, strategies or opinions. All content provided on this blog is for informational purposes only. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. The owner will not be liable for any errors or omissions in this information nor for the availability of this information. The owner will not be liable for any losses, injuries, or damages from the display or use of his information. //­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­ Disclaimer Did you find this Blog helpful ?? Let me know @ www.facebook.com/datastage4you. Ethereal template. Powered by Blogger.