"mmap failed: Not enough space"

ahmedwaseem2000 · Post by **ahmedwaseem2000** » Mon Aug 18, 2008 10:03 pm

Hi,

I am getting the error when I am trying to perform a join or a lookup in a particular job. but thats not the case with all the jobs. The source data is read from a sequential file which passes the data to transformer and then the data is passed to row generator which inturn passes the data to the join/Lookup but both fail with the error message ""mmap failed: Not enough space"" Could any one please let me know what is the reason for this?

Thanks

ray.wurlod · Post by **ray.wurlod** » Mon Aug 18, 2008 11:18 pm

There's probably not enough space available at the time on whatever directory you have specified for scratch disk. Or, in the case of a Lookup stage, not enough space to load the reference data into memory.
Please post the entire (detailed) error message so we can get a better idea.

ahmedwaseem2000 · Post by **ahmedwaseem2000** » Fri Aug 22, 2008 12:26 am

Well, I was able to solve this issue. Just for the reference for the people who would be facing this issue in future. It was not the issue with the scratch disk or the swap space. But in the sort I had used "restrict memory usage" which was Causing this error. Anyways, Dsxchange is a place where in I can post when i find the end of the road.

Thanks people

ray.wurlod · Post by **ray.wurlod** » Fri Aug 22, 2008 12:47 am

You didn't mention a Sort in your original post!

Well done on resolving it yourself.

DS_MJ · Post by **DS_MJ** » Tue Jun 08, 2010 3:03 pm

ahmedwaseem2000 wrote:Well, I was able to solve this issue. Just for the reference for the people who would be facing this issue in future. It was not the issue with the scratch disk or the swap space. But in the sort I had used "restrict memory usage" which was Causing this error. Anyways, Dsxchange is a place where in I can post when i find the end of the road.

Thanks people

Hello ahmedwaseem2000:

I am getting the same error "mmap failed: Not enough space". I am using a Join stage.
Can you please let me know how you remove "restrict memory usage" in this join.

Thanks in advance.

DS_MJ · Post by **DS_MJ** » Tue Jun 08, 2010 3:05 pm

Hello ahmedwaseem2000:

I am doing a Hash partitioning in this join and not using a sort stage.
When I remove this join I dont get this error ("mmap failed: Not enough space")

Thanks again.

dsedi · Post by **dsedi** » Tue Jun 08, 2010 3:20 pm

DS_MJ wrote:Hello ahmedwaseem2000:

I am doing a Hash partitioning in this join and not using a sort stage.
When I remove this join I dont get this error ("mmap failed: Not enough space")

Thanks again.

Set the environment variable "APT_TSORT_STRESS_BLOCKSIZE" higher value and give a try..

ray.wurlod · Post by **ray.wurlod** » Tue Jun 08, 2010 5:16 pm

Note that APT_TSORT_STRESS_BLOCKSIZE, if set, overrides any Restrict Memory Usage setting in the Sort stage.

dsedi · Post by **dsedi** » Tue Jun 08, 2010 11:54 pm

ray.wurlod wrote:Note that APT_TSORT_STRESS_BLOCKSIZE, if set, overrides any Restrict Memory Usage setting in the Sort stage. ...

Ray,Thanks for looking..
"DS_MJ" said he dont have SORT stage in his design but still getting the above said Error.i told him to set the variable and verify whether that works or not.

Thanks again.

ray.wurlod · Post by **ray.wurlod** » Wed Jun 09, 2010 12:21 am

Yes, but ahmedwaseem2000 did mention a Sort stage.

DS_MJ · Post by **DS_MJ** » Wed Jun 09, 2010 8:53 am

dsedi wrote:
ray.wurlod wrote:Note that APT_TSORT_STRESS_BLOCKSIZE, if set, overrides any Restrict Memory Usage setting in the Sort stage. ...
Ray,Thanks for looking..
"DS_MJ" said he dont have SORT stage in his design but still getting the above said Error.i told him to set the variable and verify whether that works or not.

Thanks again.

Thank you all, so much, appreciate your responses.
However, I had a aggregator stage before the join stage. I removed the sort in the aggregator stage and now I dont get this error.

Thanks and Best Regards,
Mamta

zulfi123786 · Post by **zulfi123786** » Mon Jan 17, 2011 7:06 am

I too was facing this mmap issue with a job where a join stage has 32 links each with an inline sort, The most curious thing is that this job ran fine in our Development and Testing environment but failed in Production. when the operator combination was disabled it ran fine in Production

Very curious to know why ?

jwiles · Post by **jwiles** » Mon Jan 17, 2011 7:57 am

zulfi123786 wrote:I too was facing this mmap issue with a job where a join stage has 32 links each with an inline sort, The most curious thing is that this job ran fine in our Development and Testing environment but failed in Production. when the operator combination was disabled it ran fine in Production

Very curious to know why ?

32 links into a Join is unusual. In your case, Orchestrate was likely combining most/all of the sort operators (added by the inline sort) with the join operator into a single process ("Combined Operator"), and that process ran out of addressable memory. When you disabled operator combination, each sort was run as a separate process with it's own memory space.

"Inline" sort and Sort stage both utilize the tsort operator. Inline sort just provides a more convenient way to add a sort to a link, but provides fewer options to the developer.

Regards,

zulfi123786 · Post by **zulfi123786** » Mon Jan 17, 2011 9:06 am

jwiles wrote: Orchestrate was likely combining most/all of the sort operators (added by the inline sort) with the join operator into a single process ("Combined Operator"), and that process ran out of addressable memory. When you disabled operator combination, each sort was run as a separate process with it's own memory space.

Regards,

Does it mean all inline sorts together form one tsort oerator ?
and what is the difference in terms of memory usage if operators are combined into one as the amount of data is still the same

jwiles · Post by **jwiles** » Mon Jan 17, 2011 10:43 am

zulfi123786 wrote:Does it mean all inline sorts together form one tsort oerator ?

No, it just means that one process (per partitiion) is performing all of the work. Each sort is still a separate "entity".

Actually, for multiple input streams the sorts may not be combined as I am thinking. I'll need to verify the combination rules to be certain, but you can see for yourself by enabling the APT_DUMP_SCORE environment variable and examining the job score for combined operators.

and what is the difference in terms of memory usage if operators are combined into one as the amount of data is still the same

It's hard to quantify beyond some generalizations as it partially depends upon how the operating system allocates memory for individual processes. Also, each operator may have it's own memory requirements beyond normal buffering/link (each tsort, for example, allocates memory for sorting workspace). This gets into a deeply technical area which is really beyond the scope of this forum and really requires a good understanding of how the parallel engine operates.

Regards,

DSXchange