DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
This topic has been marked "Resolved."
Author Message
metadatamen



Group memberships:
Premium Members

Joined: 26 Oct 2016
Posts: 42

Points: 438

Post Posted: Thu Jun 29, 2017 1:05 pm Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
We are facing a problem when populating custom attribute values for Data File and Data File Fields assets throug istool import and "Import Asser Values"
And our observation so far is because of the "Data File Folder" section.


1. Am able to upload "File1.csv” Data file successfully with folder created separately. so the Hierarchy is Host=>Data File Folders=>Data Files=>Data File Records=>Data File Field
2. Populated the custom attribute for "Data File" and "Data File Field Type" manually in IGC and exported the “Asset Values” files to understand the syntax I should follow to import through istools.
3. Created the Actual Files to populate "Custom Attributes" for Data File and Data File Filed with exact same format as exported from Step 2 with not having Asset ID in it.

And the error am getting is as below but no actual errors captured in the log files.:

Import Completed with failures.
An error occurred and no assets were imported. For more details, check the server logs.

Sample Data :

+++ Data File - begin +++
Name,Host,Data File Folder,Data File Folder,Data File Folder,Data File Folder,Data File Folder,Short Description,Long Description,Assigned To Terms,Labels,Stewards,Implements Rules,Governed by Rules,Test
"StandardizationFile.csv","ulvsjdd01.devfg.rbc.com","data","etl","NAS","SJD0","UMI","This is FullLoadSecPos data from MILVUS (B201) which located in MIT's HDFS RAW ZONE",,,,,,,"Test"
+++ Data File - end +++

Please advise how we could fix this problem, this is really breaking our head and I can't find any supporting document to fix this.
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54375
Location: Sydney, Australia
Points: 294942

Post Posted: Fri Jun 30, 2017 5:55 am Reply with quote    Back to top    

Have you tried using a pathname as the value for Data File Folder ?

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne
Rate this response:  
Not yet rated
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3781

Points: 30365

Post Posted: Fri Jun 30, 2017 7:58 am Reply with quote    Back to top    

The folder structure for Data Files is very hard to use in identities. Try using the AssetID instead. I haven't tested for Data Files, but when this feature first became available a year or so ago ...

_________________
Ernie Ostic

blogit!
Open IGC is Here!
Rate this response:  
Not yet rated
metadatamen



Group memberships:
Premium Members

Joined: 26 Oct 2016
Posts: 42

Points: 438

Post Posted: Fri Jun 30, 2017 11:01 am Reply with quote    Back to top    

Hi Ernie,

In IBM knowledge center, I can see this comment "Asset ID. The asset ID is not required but can improve import performance."

https://www.ibm.com/support/knowledgecenter/en/SSZJPZ_11.5.0/com.ibm.swg.im.iis.bg.doc/topics/r_CSVfileFormat_info_assets.html#r_CSVfileFormat_info_assets


It is working with Asset ID, but it would be really hard to manage with Asset ID. We don't want to mention Asset ID unless it's really necessary. The coding part would be really complex otherwise.

I tried all possibilities that you have suggested but no luck.
Rate this response:  
Not yet rated
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3781

Points: 30365

Post Posted: Fri Jun 30, 2017 1:02 pm Reply with quote    Back to top    

What does the export file look like when you dump a real one...with folders, etc.. Dump one, change it's Short Description and then re-import it. Make no changes to the identity. If that doesn't ...

_________________
Ernie Ostic

blogit!
Open IGC is Here!
Rate this response:  
Not yet rated
metadatamen



Group memberships:
Premium Members

Joined: 26 Oct 2016
Posts: 42

Points: 438

Post Posted: Fri Jun 30, 2017 1:06 pm Reply with quote    Back to top    

+++ Data File - begin +++
Name,Host,Data File Folder,Data File Folder,Data File Folder,Data File Folder,Data File Folder,Short Description,Long Description,Assigned To Terms,Labels,Stewards,Implements Rules,Governed by Rules,Test
"StandardizationFile.csv","ulvsjdd01.devfg.rbc.com","data","etl","NAS","SJD0","UMI","This is FullLoadSecPos data from MILVUS (B201) which located in MIT's HDFS RAW ZONE",,,,,,,"Test"
+++ Data File - end +++


Please see above, That's exactly I did, just that I removed AssetID field, everything else was just exported as is.

This seems to be a bug, I will have a call with IBM Support next week and see how it goes. I tried with all possibilities but nothing worked. We have the same problem for Data Field import also when having Data File Folder defined.
Rate this response:  
Not yet rated
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3781

Points: 30365

Post Posted: Fri Jun 30, 2017 2:12 pm Reply with quote    Back to top    

Try also, one more thing.....test it with just removing the value of the Asset Id field.....not the comma placeholder or its heading. Leave those in. Just in case it is an issue with the header. ...

_________________
Ernie Ostic

blogit!
Open IGC is Here!
Rate this response:  
Not yet rated
metadatamen



Group memberships:
Premium Members

Joined: 26 Oct 2016
Posts: 42

Points: 438

Post Posted: Fri Jun 30, 2017 2:33 pm Reply with quote    Back to top    

Tried as below but same issue.

+++ Data File - begin +++
Name,Host,Data File Folder,Data File Folder,Data File Folder,Data File Folder,Data File Folder,Asset Id,Short Description,Long Description
"ArchiveFile.csv","ulvsjdd01.devfg.rbc.com","data","etl","NAS","SJD0","UMI",,"This is an Archive File",
+++ Data File - end +++
Rate this response:  
Not yet rated
metadatamen



Group memberships:
Premium Members

Joined: 26 Oct 2016
Posts: 42

Points: 438

Post Posted: Tue Jul 18, 2017 1:56 pm Reply with quote    Back to top    

Ok this is a defect in IGC.

When we have Data File Folder in Hierarchy, we will not be able to populate Custom Attributes or create lineage.

IBM is working to fix this issue.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours