Unit tests consuming a lot of disk
Locked
Unit tests consuming a lot of disk
 
|
Hi All,
I get *Caused by: java.io.IOException: No space left on device* errors from some tests when running the flink unit tests on my mac. I have 30 GB free space on my machine and I am building the latest code from the master branch. The following tests in flink-runtime are failing with this error [INFO] Results: [INFO] [ERROR] Errors: [ERROR] SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultBoth:91->submitJobGraphAndWait:97 » JobExecution [ERROR] SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultReceiver:84->submitJobGraphAndWait:97 » JobExecution [ERROR] SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultSender:77->submitJobGraphAndWait:97 » JobExecution [ERROR] ScheduleOrUpdateConsumersTest.testMixedPipelinedAndBlockingResults:128 » JobExecution I tried reducing the test parallelism with -Dflink.forkCount=2 , however that did not help. I'm confident that the tests are the issue since I can see disk usage increase in real-time as I run the tests. After the tests complete, the disk usage decreases. Is this a known issue? Or would this be something worth investigating as an improvement? Thanks, Tim |
Re: Unit tests consuming a lot of disk
|
|
Hi Tim,
I wouldn’t expect these tests to consume 30GB of space. Any chance your temp dir is using a mount point with much less free space? — Ken > On Jun 14, 2019, at 12:28 PM, Timothy Farkas <[hidden email]> wrote: > > Hi All, > > I get *Caused by: java.io.IOException: No space left on device* errors from > some tests when running the flink unit tests on my mac. I have 30 GB free > space on my machine and I am building the latest code from the master > branch. The following tests in flink-runtime are failing with this error > > [INFO] Results: > > [INFO] > > [ERROR] Errors: > > [ERROR] SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultBoth:91->submitJobGraphAndWait:97 > » JobExecution > > [ERROR] SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultReceiver:84->submitJobGraphAndWait:97 > » JobExecution > > [ERROR] SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultSender:77->submitJobGraphAndWait:97 > » JobExecution > > [ERROR] ScheduleOrUpdateConsumersTest.testMixedPipelinedAndBlockingResults:128 > » JobExecution > > I tried reducing the test parallelism with -Dflink.forkCount=2 , however > that did not help. I'm confident that the tests are the issue since I can > see disk usage increase in real-time as I run the tests. After the tests > complete, the disk usage decreases. > > Is this a known issue? Or would this be something worth investigating as an > improvement? > > Thanks, > Tim -------------------------- Ken Krugler +1 530-210-6378 http://www.scaleunlimited.com Custom big data solutions & training Flink, Solr, Hadoop, Cascading & Cassandra |
Re: Unit tests consuming a lot of disk
|
|
Hi Ken,
I don't believe so, my main disk should be used to store /tmp. Timothys-MacBook-Pro:~ tfarkas$ diskutil list /dev/disk0 (internal): #: TYPE NAME SIZE IDENTIFIER 0: GUID_partition_scheme 251.0 GB disk0 1: EFI EFI 314.6 MB disk0s1 2: Apple_CoreStorage Macintosh HD 250.0 GB disk0s2 3: Apple_Boot Recovery HD 650.0 MB disk0s3 /dev/disk1 (internal, virtual): #: TYPE NAME SIZE IDENTIFIER 0: Apple_HFS Macintosh HD +249.7 GB disk1 Logical Volume on disk0s2 DA9C82BE-D97D-4D65-8166-9F742F9AC884 Unencrypted Timothys-MacBook-Pro:~ tfarkas$ mount /dev/disk1 on / (hfs, local, journaled) devfs on /dev (devfs, local, nobrowse) map -hosts on /net (autofs, nosuid, automounted, nobrowse) map auto_home on /home (autofs, automounted, nobrowse) Thanks, Tim On Fri, Jun 14, 2019 at 12:33 PM Ken Krugler <[hidden email]> wrote: > Hi Tim, > > I wouldn’t expect these tests to consume 30GB of space. > > Any chance your temp dir is using a mount point with much less free space? > > — Ken > > > On Jun 14, 2019, at 12:28 PM, Timothy Farkas <[hidden email]> > wrote: > > Hi All, > > I get *Caused by: java.io.IOException: No space left on device* errors from > some tests when running the flink unit tests on my mac. I have 30 GB free > space on my machine and I am building the latest code from the master > branch. The following tests in flink-runtime are failing with this error > > [INFO] Results: > > [INFO] > > [ERROR] Errors: > > [ERROR] > SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultBoth:91->submitJobGraphAndWait:97 > » JobExecution > > [ERROR] > SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultReceiver:84->submitJobGraphAndWait:97 > » JobExecution > > [ERROR] > SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultSender:77->submitJobGraphAndWait:97 > » JobExecution > > [ERROR] > ScheduleOrUpdateConsumersTest.testMixedPipelinedAndBlockingResults:128 > » JobExecution > > I tried reducing the test parallelism with -Dflink.forkCount=2 , however > that did not help. I'm confident that the tests are the issue since I can > see disk usage increase in real-time as I run the tests. After the tests > complete, the disk usage decreases. > > Is this a known issue? Or would this be something worth investigating as an > improvement? > > Thanks, > Tim > > > -------------------------- > Ken Krugler > +1 530-210-6378 > http://www.scaleunlimited.com > Custom big data solutions & training > Flink, Solr, Hadoop, Cascading & Cassandra > > |
Re: Unit tests consuming a lot of disk
|
|
Resolved the issue. I was running a very old version of macos, after
upgrading to Mojave the issue disappeared. Disk usage stopped spiking and I stopped running out of disk space. I'm guessing there was a bug with how macos used to handle swap space or memory mapped files that was later fixed. Shame on me for being delinquent with my upgrades :) . Thanks for the help Ken. Tim On Fri, Jun 14, 2019 at 12:50 PM Timothy Farkas < [hidden email]> wrote: > Hi Ken, > > I don't believe so, my main disk should be used to store /tmp. > > Timothys-MacBook-Pro:~ tfarkas$ diskutil list > > /dev/disk0 (internal): > > #: TYPE NAME SIZE > IDENTIFIER > > 0: GUID_partition_scheme 251.0 GB disk0 > > 1: EFI EFI 314.6 MB > disk0s1 > > 2: Apple_CoreStorage Macintosh HD 250.0 GB > disk0s2 > > 3: Apple_Boot Recovery HD 650.0 MB > disk0s3 > > > /dev/disk1 (internal, virtual): > > #: TYPE NAME SIZE > IDENTIFIER > > 0: Apple_HFS Macintosh HD +249.7 GB disk1 > > Logical Volume on disk0s2 > > DA9C82BE-D97D-4D65-8166-9F742F9AC884 > > Unencrypted > > > Timothys-MacBook-Pro:~ tfarkas$ mount > > /dev/disk1 on / (hfs, local, journaled) > > devfs on /dev (devfs, local, nobrowse) > > map -hosts on /net (autofs, nosuid, automounted, nobrowse) > > map auto_home on /home (autofs, automounted, nobrowse) > > > > Thanks, > > Tim > > On Fri, Jun 14, 2019 at 12:33 PM Ken Krugler <[hidden email]> > wrote: > >> Hi Tim, >> >> I wouldn’t expect these tests to consume 30GB of space. >> >> Any chance your temp dir is using a mount point with much less free space? >> >> — Ken >> >> >> On Jun 14, 2019, at 12:28 PM, Timothy Farkas < >> [hidden email]> wrote: >> >> Hi All, >> >> I get *Caused by: java.io.IOException: No space left on device* errors >> from >> some tests when running the flink unit tests on my mac. I have 30 GB free >> space on my machine and I am building the latest code from the master >> branch. The following tests in flink-runtime are failing with this error >> >> [INFO] Results: >> >> [INFO] >> >> [ERROR] Errors: >> >> [ERROR] >> SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultBoth:91->submitJobGraphAndWait:97 >> » JobExecution >> >> [ERROR] >> SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultReceiver:84->submitJobGraphAndWait:97 >> » JobExecution >> >> [ERROR] >> SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultSender:77->submitJobGraphAndWait:97 >> » JobExecution >> >> [ERROR] >> ScheduleOrUpdateConsumersTest.testMixedPipelinedAndBlockingResults:128 >> » JobExecution >> >> I tried reducing the test parallelism with -Dflink.forkCount=2 , however >> that did not help. I'm confident that the tests are the issue since I can >> see disk usage increase in real-time as I run the tests. After the tests >> complete, the disk usage decreases. >> >> Is this a known issue? Or would this be something worth investigating as >> an >> improvement? >> >> Thanks, >> Tim >> >> >> -------------------------- >> Ken Krugler >> +1 530-210-6378 >> http://www.scaleunlimited.com >> Custom big data solutions & training >> Flink, Solr, Hadoop, Cascading & Cassandra >> >> |
Re: Unit tests consuming a lot of disk
|
|
Good to know that you have solved your problem :)
Piotrek > On 15 Jun 2019, at 00:48, Timothy Farkas <[hidden email]> wrote: > > Resolved the issue. I was running a very old version of macos, after > upgrading to Mojave the issue disappeared. Disk usage stopped spiking and I > stopped running out of disk space. I'm guessing there was a bug with how > macos used to handle swap space or memory mapped files that was later > fixed. Shame on me for being delinquent with my upgrades :) . > > Thanks for the help Ken. > Tim > > On Fri, Jun 14, 2019 at 12:50 PM Timothy Farkas < > [hidden email]> wrote: > >> Hi Ken, >> >> I don't believe so, my main disk should be used to store /tmp. >> >> Timothys-MacBook-Pro:~ tfarkas$ diskutil list >> >> /dev/disk0 (internal): >> >> #: TYPE NAME SIZE >> IDENTIFIER >> >> 0: GUID_partition_scheme 251.0 GB disk0 >> >> 1: EFI EFI 314.6 MB >> disk0s1 >> >> 2: Apple_CoreStorage Macintosh HD 250.0 GB >> disk0s2 >> >> 3: Apple_Boot Recovery HD 650.0 MB >> disk0s3 >> >> >> /dev/disk1 (internal, virtual): >> >> #: TYPE NAME SIZE >> IDENTIFIER >> >> 0: Apple_HFS Macintosh HD +249.7 GB disk1 >> >> Logical Volume on disk0s2 >> >> DA9C82BE-D97D-4D65-8166-9F742F9AC884 >> >> Unencrypted >> >> >> Timothys-MacBook-Pro:~ tfarkas$ mount >> >> /dev/disk1 on / (hfs, local, journaled) >> >> devfs on /dev (devfs, local, nobrowse) >> >> map -hosts on /net (autofs, nosuid, automounted, nobrowse) >> >> map auto_home on /home (autofs, automounted, nobrowse) >> >> >> >> Thanks, >> >> Tim >> >> On Fri, Jun 14, 2019 at 12:33 PM Ken Krugler <[hidden email]> >> wrote: >> >>> Hi Tim, >>> >>> I wouldn’t expect these tests to consume 30GB of space. >>> >>> Any chance your temp dir is using a mount point with much less free space? >>> >>> — Ken >>> >>> >>> On Jun 14, 2019, at 12:28 PM, Timothy Farkas < >>> [hidden email]> wrote: >>> >>> Hi All, >>> >>> I get *Caused by: java.io.IOException: No space left on device* errors >>> from >>> some tests when running the flink unit tests on my mac. I have 30 GB free >>> space on my machine and I am building the latest code from the master >>> branch. The following tests in flink-runtime are failing with this error >>> >>> [INFO] Results: >>> >>> [INFO] >>> >>> [ERROR] Errors: >>> >>> [ERROR] >>> SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultBoth:91->submitJobGraphAndWait:97 >>> » JobExecution >>> >>> [ERROR] >>> SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultReceiver:84->submitJobGraphAndWait:97 >>> » JobExecution >>> >>> [ERROR] >>> SlotCountExceedingParallelismTest.testNoSlotSharingAndBlockingResultSender:77->submitJobGraphAndWait:97 >>> » JobExecution >>> >>> [ERROR] >>> ScheduleOrUpdateConsumersTest.testMixedPipelinedAndBlockingResults:128 >>> » JobExecution >>> >>> I tried reducing the test parallelism with -Dflink.forkCount=2 , however >>> that did not help. I'm confident that the tests are the issue since I can >>> see disk usage increase in real-time as I run the tests. After the tests >>> complete, the disk usage decreases. >>> >>> Is this a known issue? Or would this be something worth investigating as >>> an >>> improvement? >>> >>> Thanks, >>> Tim >>> >>> >>> -------------------------- >>> Ken Krugler >>> +1 530-210-6378 >>> http://www.scaleunlimited.com >>> Custom big data solutions & training >>> Flink, Solr, Hadoop, Cascading & Cassandra >>> >>> |
«
Return to (DEPRECATED) Apache Flink Mailing List archive.
|
1 view|%1 views
| Free forum by Nabble | Edit this page |