How to Optimize Tmap Component in Talend

Optimize Tmap component in Talend


If Lookup table has few rows:

• Open tmap select lookup model as Load Once .This will load lookup data one time before mail flow starts.
• This Lookup data will be stored in memory then main flow execution is very fast with comparing with lookup data in memory.

IF lookup table data is very large:
Talend cannont store the lookup table data in memory. you will get java heap space Execption. To resolve this issue follow the below steps

• open tmap
• go to lookup table
• click on tmap settings
• select the value for store temp data property to True 
• click on ok
• In Tmap properties basic settings  set Temp Data Directory path by browsing folder
• Go to Advance settings
• set  max buffer size(nb of row) to some value based on  lookup condition data type

In my jobs lookup stressful for  2 mill records with 3 GB of memory  of integer data type.

Exception in component tFileOutputDelimited

Error: 
Exception in component tFileOutputDelimited_1
java.io.FileNotFoundException: C:/testfile.csv (The process cannot access the file because it is being used by another process)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(Unknown Source)
at java.io.FileOutputStream.<init>(Unknown Source)

Solution:1  If file is already Available in that location

• This file is already opened. You need to close the file and execute this job again.

Solution:2  If file not exist

•  Go to components --> go to advanced settings -->  check create directory if not exists property

Then Execute the job. file will be created along with directory structure.

ETL Talend Training in Hyderabad

Contact me on the below number for training on Talend ETL tool... ;)
- Raja
Talend Trainer
+91-924-736-0004