I have a flat files source and it does'nt have any key to comapre record(Like business key).How i can perform incremental loading to the corresponding target.(targate is also a flat file).please post needfull.
thanks in advance,
satish dubey
How can perform incremental loading without using key
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 28
- Joined: Mon Nov 28, 2005 1:23 am
Satish,
The solution I am suggesting is on the basis that the following understanding is correct. Namely, that you have two files:
A: The "master" which represents the current state and
B: The file holding records that may have potentially changed
You want to take only those records in B that are not in A and append them to A.
This solution is based on UNIX commands (I'm like a scratched record!).
The solution sorts file A, then sorts file B and then appends to the existing A those that are in the sorted B that are not already in the sorted A.
Is this something along the lines of what you are looking for? If it is close but you want to do something like removing the records that are in A but not in B (i.e. the "old" records") then a variation on the comm command would be required. "man comm" for more details.
David.
The solution I am suggesting is on the basis that the following understanding is correct. Namely, that you have two files:
A: The "master" which represents the current state and
B: The file holding records that may have potentially changed
You want to take only those records in B that are not in A and append them to A.
This solution is based on UNIX commands (I'm like a scratched record!).
Code: Select all
sort A > A.sorted
sort B > B.sorted
comm -13 A.sorted B.sorted >> A
Is this something along the lines of what you are looking for? If it is close but you want to do something like removing the records that are in A but not in B (i.e. the "old" records") then a variation on the comm command would be required. "man comm" for more details.
David.
(Previously known as D)
Be alturistic and donate your spare CPU cycles to research. http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1
Be alturistic and donate your spare CPU cycles to research. http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1