DataSet rejecting null records
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 296
- Joined: Sun Nov 16, 2008 7:41 pm
DataSet rejecting null records
Hi All
I am reading the data from a table and creating a dataset. Later i will use the same dataset in another job, do some transformations creating another file.
When i read the dataset in second job ( which is created in first job), some of the records are getting rejected because of a null in one the column.
Error below:
APT_CombinedOperatorController(2),1: Field 'HENCL' from input dataset '0' is NULL. Record dropped.
I know dataset accept nulls. I have checked the metadata and nullability was set to Yes. I couldn't figure why dataset is not reading the record with HENCL column NULL. I want this column as null.
Please help me out.
I am reading the data from a table and creating a dataset. Later i will use the same dataset in another job, do some transformations creating another file.
When i read the dataset in second job ( which is created in first job), some of the records are getting rejected because of a null in one the column.
Error below:
APT_CombinedOperatorController(2),1: Field 'HENCL' from input dataset '0' is NULL. Record dropped.
I know dataset accept nulls. I have checked the metadata and nullability was set to Yes. I couldn't figure why dataset is not reading the record with HENCL column NULL. I want this column as null.
Please help me out.
* What database are you coming from?
* How is the data being read from the database? (SQL, table read, etc.)
* How is the field defined in the database?
* Have you confirmed that the data is stored in teh dataset as nullable? (Note: I like to look at the dataset on the server itself and check its schema using orchadmin: orchadmin describe -s <dataset>)
Brad.
* How is the data being read from the database? (SQL, table read, etc.)
* How is the field defined in the database?
* Have you confirmed that the data is stored in teh dataset as nullable? (Note: I like to look at the dataset on the server itself and check its schema using orchadmin: orchadmin describe -s <dataset>)
Brad.
It is not that I am addicted to coffee, it's just that I need it to survive.
-
- Premium Member
- Posts: 296
- Joined: Sun Nov 16, 2008 7:41 pm
Column in database is nullable. I am using user defined SQL to pull data form database. I want this column as null. Thats why i am not using any null functions on this column. I cannot check orchadmin as it is not installed properly and not working. Is there any other way i can check teh descriptor file of datase.
Not to get distracted, but when you say orchadmin is not installed right or not working, are you getting library errors? Something like this:
If so, do the following at the command line:
Then, in the same session where the dsenv was sourced, do your orachadmin commands. I know that on our system, I have to source in dsenv before I can run DataStage commands.
Brad.
??exec(): 0509-036 Cannot load program orchadmin because of the following errors:
0509-150 Dependent module /opt/CA/UnicenterAutoSysJM/autosys/lib/libicui18n.a could not be loaded.
0509-103 The module has an invalid magic number.
If so, do the following at the command line:
Code: Select all
. $DSHOME/dsenv
Brad.
It is not that I am addicted to coffee, it's just that I need it to survive.
First, rerun the job with APT_DISABLE_COMBINATION set to True and then let us know the exact operator/stage producing the error.
Mike Hester
mhester@petra-ps.com
mhester@petra-ps.com
-
- Premium Member
- Posts: 296
- Joined: Sun Nov 16, 2008 7:41 pm
I did ran the job with the paramter and log says Dataset is producing the erorr. Does DataSets doesnt take NULLS.mhester wrote:First, rerun the job with APT_DISABLE_COMBINATION set to True and then let us know the exact operator/stage producing the error.
Whenever a look up is failed for a column, i am getting blank as the output instead of null. I was using NullTovalue(nullcolumn) and its not working. But when i changed to nullcolumn='', it started working.
When you use NullToValue function you have to give some default value. Usually NullToValue(Column,'') if you want to default it to empty string.
As far as your original error message, are you doing any transformation on the field HENCL like using a Trim function etc without null handling.
These warnings are thrown whenever you are trying to apply a transform function without handling null values explicitly because a null value cannot be trimmed etc.
Hope that helps.
As far as your original error message, are you doing any transformation on the field HENCL like using a Trim function etc without null handling.
Code: Select all
APT_CombinedOperatorController(2),1: Field 'HENCL' from input dataset '0' is NULL. Record dropped.
Hope that helps.
Kris
Where's the "Any" key?-Homer Simpson
Where's the "Any" key?-Homer Simpson