DSXchange

it should be easy enough to know if you wrote any data and if not, delete the file?

validate it with a Visual basic transformer, just convert each byte of the string to its numeric format with | between.

Char and convert are not supporting for the characters whose ASCII is greater than 256. What to do in this case? not sure about datastage but 'most' software if told a 'text' file is in 1-byte code (ascii or one-off encodings) it will read unicode letters as a group of multiple ascii symbols. For e...

I shamefully do not know what tool generates our HTML models. I can find out. Its *probably* an IBM tool, since we are heavily invested there, but it could be something else entire. Unrelated.. Regular notepad chugs up on large files. The Gun version of this does not, so I use that one sometimes. N+...

SSD was me thinking of dumping the file into ram as fast as possible. I think the lookup stage is doing something more beyond just a grab --- it may not be the disk read time, but the tool, if it took 45 min. I mean you should be able to read a gig every couple of seconds, and a 10gb file should dro...

NICE, I looked but did not find this list.
Looks like the one I was using had at least 5 offending characters in it..

Also seems like login to the tool does not matter what you have, but inside jobs it will cause troubles.

ok, that is what I understood also. If you are OK going outside of datastage and databases, sure, its very simple to make a C program do this, again looking at a memory mapped file, or eating up a bunch of ram on one box, or maybe doing a fixed record random access file on a huge (pricey) flash driv...

We use FastTrack for mapping columns and the rules that define the mapping. Nothing else, but its been great for just that one task of keeping the mapping public, up to date, centralized, etc. The tool is terribly slow, but that is becoming the norm, opening Excel in Office 365 takes 10-20 times lon...

this could be TOTALLY unrelated and off the wall, but let me describe a recent issue I was having... I was having my password constantly rejected for a teradata system. My password was rather lengthy and about 50% special characters (standard USA keyboard punctuation, math, and similar stuff like !@...

what does non-database mean?

you can do a lookup in a 'binary' fixed-width file with a hashing algorithm.
If it will fit, you can do that in memory even faster, or a memory mapped file.

but we are way, way outside of datastage coding at this point...

if dealing with 'very' large volumes and few failed lookups you may find it more efficient to reject them and only transform the reject link, then funnel back. But generally, the above is the best way. Its more stages to reject but less actual work, as far as I can tell.

I think that is all of them?
I only see a few also...

have you done stats and index on the target? Inserts can go long if the stats are a mess, or if the table isnt indexed well for the job. bulk works well too, if you go that route. Hardware and systems vary so its hard to say where to do a cutoff of inserts vs bulk mode. What seems few to a big syste...

you can make the first part unique and the back end of the job reusable to protect your metadata. The front piece would do any manual conversions and corrections. Or you could make the RCP version, validate your tables, and if any are incorrect write unique jobs for only those tables? you can also m...

Hey experts, I am having a little trouble trying to find the magic combination of settings for the MQ connector stage that allows it to 'do nothing' if there are no messages on the queue. What I have is a batch process that periodically will empty a queue that is connected to a topic. It is unusual ...