DSXchange

Define the columns in the stage to match the data columns in the file, the records from 11 on. The stage will automatically discard the first 10 for not matching that.

:? Confusing. This second OCI stage that generates your error, what exactly is it doing - selecting from a source or writing to a target? My assumption is that your error is caused by your use of a 'custom query' without understanding the rules that the OCI stage enforces. Always best to let the sta...

For whatever reason, your example is not making any sense to me this morning. However, sure seems like the join for something like this could be best done in the source query rather than via some complex lookup. To discuss this more, seems to me that real examples with actual values, both source and...

Yes.

What are you using the CRC32 value for, why are you generating it? If you are thinking it will create a unique surrogate key, that is not the right thing to be using it for.

And what ".log files" would that be?

What makes you think you've got it in the wrong spot? Ray did say that the keyword would cause a failure if it couldn't be used. If you still think you've got a syntax problem, post the error you received. A 'trick' to try would be to declare three keys rather than your five when building the hashed...

You use TO_CHAR() when selecting, not TO_DATE() as it already is one in Oracle as thus only used when loading data into a DATE field. Then the appropriate StringTo?? function can convert it to whatever you need, date or timestamp, inside the job depending if you need the time portion or not. And los...

Yes, it makes a HUGE difference and was something we discussed. Your "file not found" error comes from the "ls -m" you've mysteriously decided to leave at the end of your command. As noted much earlier, get rid of it. Problem solved.

OK. There was a couple of reasons why, after much pondering, I made that comment. And since you asked understand this is all me, my personal opinion so take it cum grano salis as Ray might say. :lol: First off I agree with your comment about matches being 'syntactically more elegant' and would vastl...

There are separate versions of DataStage specific to Windows and to each supported flavor of UNIX. So just like anything else, the same software cannot run on both.

You created what index? How, exactly?

.....by building an "API layer" between the source and your ETL..............views built in the source system.......... Hmmm!! How we like to have our way in the source systems, don't we? Wouldn't we love it if they had a column in each source table to tell us if/when the row was updated?...

You've already got that functionality with the UserVariables and Start/End Loop stages. You manually built something that it could have handled on its own as there's no need to count in your case. Simply use something in the UserVariables stage to supply the contents of the file in a 'delimited list...

Yes, you'd just make it more... difficult... then you'd have to worry about unique keys and destructive overwrite and all that. Stick with the flat file.

Seems like much of this could be addressed by building an "API layer" between the source and your ETL. That could be something as simple as views built in the source system to establish your target column names and types and you use them as what you actually source from. Then changes to th...