Hi everybody,
as I've recently started contributing to Flink, I'd like to ask you a few tips on how you normally run the tests to ensure that everything runs smoothly before pushing the code to CI. First, I'd like to point out two issues in particular I had: 1. some test suites (KMeansWithBroadcastSetITCase.java, ScalaCsvReaderWithPOJOITCase.scala and CsvReaderITCase.java) kept failing because the expected results (string literals) were matched against an object rendered as a string using the String.format method, a method whose result depends on the default Locale; as my Locale (Italian) renders doubles with a comma instead of a dot as the decimal separator, the representation of doubles diverged from the expected one, thus making my tests fail; I've solved it locally by explicitly using the US locale for those calls — I'm thinking about opening an issue and submitting my solution as a PR; 2. I had another problem with WebFrontendITCase.java: the test suite starts a mini cluster with the web server on port 8081; not knowing this and (unknowingly) having another process listening on that port, the test suite kept failing at initialization (line 59); however, the message regarding the port being busy was not displayed, despite being logged at ERROR level (WebMonitorUtils.startWebRuntimeMonitor, line 138); what configuration file should I edit in order for these issues to come up instantly without having to dig into the code to understand what's happening? Apart from these issues in particular, I'd like to ask if you have any particular routine when it comes to building and testing Flink locally. I'm not particularly knowledgable regarding Maven, so right now I'm basically running `mvn [clean] install [-rf :<last-failed-module>]` each time to make sure I have a fresh build to test. Would this be the right path or is there a quicker way to have a fresh build and running the tests on them? Feel free to point me to any relevant documentation, if you wish. Thank you in advance for any tip. :) -- BR, Stefano Baghino Software Engineer @ Radicalbit |
Hi!
I basically test with "mvn clean install" as well. Greetings, Stephan On Sat, Feb 6, 2016 at 10:31 PM, Stefano Baghino < [hidden email]> wrote: > Hi everybody, > > as I've recently started contributing to Flink, I'd like to ask you a few > tips on how you normally run the tests to ensure that everything runs > smoothly before pushing the code to CI. > > First, I'd like to point out two issues in particular I had: > > 1. some test suites (KMeansWithBroadcastSetITCase.java, > ScalaCsvReaderWithPOJOITCase.scala and CsvReaderITCase.java) kept > failing > because the expected results (string literals) were matched against an > object rendered as a string using the String.format method, a method > whose > result depends on the default Locale; as my Locale (Italian) renders > doubles with a comma instead of a dot as the decimal separator, the > representation of doubles diverged from the expected one, thus making my > tests fail; I've solved it locally by explicitly using the US locale for > those calls — I'm thinking about opening an issue and submitting my > solution as a PR; > 2. I had another problem with WebFrontendITCase.java: the test suite > starts a mini cluster with the web server on port 8081; not knowing this > and (unknowingly) having another process listening on that port, the > test > suite kept failing at initialization (line 59); however, the message > regarding the port being busy was not displayed, despite being logged at > ERROR level (WebMonitorUtils.startWebRuntimeMonitor, line 138); what > configuration file should I edit in order for these issues to come up > instantly without having to dig into the code to understand what's > happening? > > Apart from these issues in particular, I'd like to ask if you have any > particular routine when it comes to building and testing Flink locally. I'm > not particularly knowledgable regarding Maven, so right now I'm basically > running `mvn [clean] install [-rf :<last-failed-module>]` each time to make > sure I have a fresh build to test. Would this be the right path or is there > a quicker way to have a fresh build and running the tests on them? Feel > free to point me to any relevant documentation, if you wish. > > Thank you in advance for any tip. :) > > -- > BR, > Stefano Baghino > > Software Engineer @ Radicalbit > |
Hi Stefano,
1) Please open a pull request. If the String depends on the locale, this looks like a bug. 2) "mvn clean install" is the way to go for the complete check. If you only want to run certain tests, selecting and running them from IntelliJ works pretty well. In addition, it is nice to push a version of your changes to Travis which builds it for you in the background. It takes longer to build then on a fast machine though. Cheers, Max On Sun, Feb 7, 2016 at 7:12 PM, Stephan Ewen <[hidden email]> wrote: > Hi! > > I basically test with "mvn clean install" as well. > > Greetings, > Stephan > > > On Sat, Feb 6, 2016 at 10:31 PM, Stefano Baghino < > [hidden email]> wrote: > >> Hi everybody, >> >> as I've recently started contributing to Flink, I'd like to ask you a few >> tips on how you normally run the tests to ensure that everything runs >> smoothly before pushing the code to CI. >> >> First, I'd like to point out two issues in particular I had: >> >> 1. some test suites (KMeansWithBroadcastSetITCase.java, >> ScalaCsvReaderWithPOJOITCase.scala and CsvReaderITCase.java) kept >> failing >> because the expected results (string literals) were matched against an >> object rendered as a string using the String.format method, a method >> whose >> result depends on the default Locale; as my Locale (Italian) renders >> doubles with a comma instead of a dot as the decimal separator, the >> representation of doubles diverged from the expected one, thus making my >> tests fail; I've solved it locally by explicitly using the US locale for >> those calls — I'm thinking about opening an issue and submitting my >> solution as a PR; >> 2. I had another problem with WebFrontendITCase.java: the test suite >> starts a mini cluster with the web server on port 8081; not knowing this >> and (unknowingly) having another process listening on that port, the >> test >> suite kept failing at initialization (line 59); however, the message >> regarding the port being busy was not displayed, despite being logged at >> ERROR level (WebMonitorUtils.startWebRuntimeMonitor, line 138); what >> configuration file should I edit in order for these issues to come up >> instantly without having to dig into the code to understand what's >> happening? >> >> Apart from these issues in particular, I'd like to ask if you have any >> particular routine when it comes to building and testing Flink locally. I'm >> not particularly knowledgable regarding Maven, so right now I'm basically >> running `mvn [clean] install [-rf :<last-failed-module>]` each time to make >> sure I have a fresh build to test. Would this be the right path or is there >> a quicker way to have a fresh build and running the tests on them? Feel >> free to point me to any relevant documentation, if you wish. >> >> Thank you in advance for any tip. :) >> >> -- >> BR, >> Stefano Baghino >> >> Software Engineer @ Radicalbit >> |
Hi Maximilian,
thanks for the tips. I've already opened a PR (here <https://github.com/apache/flink/pull/1598>). Is there a way from the command line to do what can be done with IntelliJ? I mean build only what's needed and run a specific test? I've tried `mvn test` but it skips the build phase and I don't know how to perform an incremental build on Maven. Thank you again! On Mon, Feb 8, 2016 at 9:49 AM, Maximilian Michels <[hidden email]> wrote: > Hi Stefano, > > 1) Please open a pull request. If the String depends on the locale, > this looks like a bug. > > 2) "mvn clean install" is the way to go for the complete check. If you > only want to run certain tests, selecting and running them from > IntelliJ works pretty well. In addition, it is nice to push a version > of your changes to Travis which builds it for you in the background. > It takes longer to build then on a fast machine though. > > Cheers, > Max > > On Sun, Feb 7, 2016 at 7:12 PM, Stephan Ewen <[hidden email]> wrote: > > Hi! > > > > I basically test with "mvn clean install" as well. > > > > Greetings, > > Stephan > > > > > > On Sat, Feb 6, 2016 at 10:31 PM, Stefano Baghino < > > [hidden email]> wrote: > > > >> Hi everybody, > >> > >> as I've recently started contributing to Flink, I'd like to ask you a > few > >> tips on how you normally run the tests to ensure that everything runs > >> smoothly before pushing the code to CI. > >> > >> First, I'd like to point out two issues in particular I had: > >> > >> 1. some test suites (KMeansWithBroadcastSetITCase.java, > >> ScalaCsvReaderWithPOJOITCase.scala and CsvReaderITCase.java) kept > >> failing > >> because the expected results (string literals) were matched against > an > >> object rendered as a string using the String.format method, a method > >> whose > >> result depends on the default Locale; as my Locale (Italian) renders > >> doubles with a comma instead of a dot as the decimal separator, the > >> representation of doubles diverged from the expected one, thus > making my > >> tests fail; I've solved it locally by explicitly using the US locale > for > >> those calls — I'm thinking about opening an issue and submitting my > >> solution as a PR; > >> 2. I had another problem with WebFrontendITCase.java: the test suite > >> starts a mini cluster with the web server on port 8081; not knowing > this > >> and (unknowingly) having another process listening on that port, the > >> test > >> suite kept failing at initialization (line 59); however, the message > >> regarding the port being busy was not displayed, despite being > logged at > >> ERROR level (WebMonitorUtils.startWebRuntimeMonitor, line 138); what > >> configuration file should I edit in order for these issues to come up > >> instantly without having to dig into the code to understand what's > >> happening? > >> > >> Apart from these issues in particular, I'd like to ask if you have any > >> particular routine when it comes to building and testing Flink locally. > I'm > >> not particularly knowledgable regarding Maven, so right now I'm > basically > >> running `mvn [clean] install [-rf :<last-failed-module>]` each time to > make > >> sure I have a fresh build to test. Would this be the right path or is > there > >> a quicker way to have a fresh build and running the tests on them? Feel > >> free to point me to any relevant documentation, if you wish. > >> > >> Thank you in advance for any tip. :) > >> > >> -- > >> BR, > >> Stefano Baghino > >> > >> Software Engineer @ Radicalbit > >> > -- BR, Stefano Baghino Software Engineer @ Radicalbit |
Hi Stefano,
Thanks for the pull request. You may use the following to only run tests from flink-runtime: > mvn verify -pl flink-runtime Alternatively, change to the flink-runtime directory and run "mvn verify" from there. This assumes Flink has been built and installed locally before. If not, Maven will try to download the artifacts. Cheers, Max On Mon, Feb 8, 2016 at 9:55 AM, Stefano Baghino <[hidden email]> wrote: > Hi Maximilian, > > thanks for the tips. I've already opened a PR (here > <https://github.com/apache/flink/pull/1598>). > Is there a way from the command line to do what can be done with IntelliJ? > I mean build only what's needed and run a specific test? I've tried `mvn > test` but it skips the build phase and I don't know how to perform an > incremental build on Maven. > > Thank you again! > > On Mon, Feb 8, 2016 at 9:49 AM, Maximilian Michels <[hidden email]> wrote: > >> Hi Stefano, >> >> 1) Please open a pull request. If the String depends on the locale, >> this looks like a bug. >> >> 2) "mvn clean install" is the way to go for the complete check. If you >> only want to run certain tests, selecting and running them from >> IntelliJ works pretty well. In addition, it is nice to push a version >> of your changes to Travis which builds it for you in the background. >> It takes longer to build then on a fast machine though. >> >> Cheers, >> Max >> >> On Sun, Feb 7, 2016 at 7:12 PM, Stephan Ewen <[hidden email]> wrote: >> > Hi! >> > >> > I basically test with "mvn clean install" as well. >> > >> > Greetings, >> > Stephan >> > >> > >> > On Sat, Feb 6, 2016 at 10:31 PM, Stefano Baghino < >> > [hidden email]> wrote: >> > >> >> Hi everybody, >> >> >> >> as I've recently started contributing to Flink, I'd like to ask you a >> few >> >> tips on how you normally run the tests to ensure that everything runs >> >> smoothly before pushing the code to CI. >> >> >> >> First, I'd like to point out two issues in particular I had: >> >> >> >> 1. some test suites (KMeansWithBroadcastSetITCase.java, >> >> ScalaCsvReaderWithPOJOITCase.scala and CsvReaderITCase.java) kept >> >> failing >> >> because the expected results (string literals) were matched against >> an >> >> object rendered as a string using the String.format method, a method >> >> whose >> >> result depends on the default Locale; as my Locale (Italian) renders >> >> doubles with a comma instead of a dot as the decimal separator, the >> >> representation of doubles diverged from the expected one, thus >> making my >> >> tests fail; I've solved it locally by explicitly using the US locale >> for >> >> those calls — I'm thinking about opening an issue and submitting my >> >> solution as a PR; >> >> 2. I had another problem with WebFrontendITCase.java: the test suite >> >> starts a mini cluster with the web server on port 8081; not knowing >> this >> >> and (unknowingly) having another process listening on that port, the >> >> test >> >> suite kept failing at initialization (line 59); however, the message >> >> regarding the port being busy was not displayed, despite being >> logged at >> >> ERROR level (WebMonitorUtils.startWebRuntimeMonitor, line 138); what >> >> configuration file should I edit in order for these issues to come up >> >> instantly without having to dig into the code to understand what's >> >> happening? >> >> >> >> Apart from these issues in particular, I'd like to ask if you have any >> >> particular routine when it comes to building and testing Flink locally. >> I'm >> >> not particularly knowledgable regarding Maven, so right now I'm >> basically >> >> running `mvn [clean] install [-rf :<last-failed-module>]` each time to >> make >> >> sure I have a fresh build to test. Would this be the right path or is >> there >> >> a quicker way to have a fresh build and running the tests on them? Feel >> >> free to point me to any relevant documentation, if you wish. >> >> >> >> Thank you in advance for any tip. :) >> >> >> >> -- >> >> BR, >> >> Stefano Baghino >> >> >> >> Software Engineer @ Radicalbit >> >> >> > > > > -- > BR, > Stefano Baghino > > Software Engineer @ Radicalbit |
Great, thank you very much!
On Monday, February 8, 2016, Maximilian Michels <[hidden email]> wrote: > Hi Stefano, > > Thanks for the pull request. > > You may use the following to only run tests from flink-runtime: > > mvn verify -pl flink-runtime > > Alternatively, change to the flink-runtime directory and run "mvn > verify" from there. > > This assumes Flink has been built and installed locally before. If > not, Maven will try to download the artifacts. > > Cheers, > Max > > On Mon, Feb 8, 2016 at 9:55 AM, Stefano Baghino > <[hidden email] <javascript:;>> wrote: > > Hi Maximilian, > > > > thanks for the tips. I've already opened a PR (here > > <https://github.com/apache/flink/pull/1598>). > > Is there a way from the command line to do what can be done with > IntelliJ? > > I mean build only what's needed and run a specific test? I've tried `mvn > > test` but it skips the build phase and I don't know how to perform an > > incremental build on Maven. > > > > Thank you again! > > > > On Mon, Feb 8, 2016 at 9:49 AM, Maximilian Michels <[hidden email] > <javascript:;>> wrote: > > > >> Hi Stefano, > >> > >> 1) Please open a pull request. If the String depends on the locale, > >> this looks like a bug. > >> > >> 2) "mvn clean install" is the way to go for the complete check. If you > >> only want to run certain tests, selecting and running them from > >> IntelliJ works pretty well. In addition, it is nice to push a version > >> of your changes to Travis which builds it for you in the background. > >> It takes longer to build then on a fast machine though. > >> > >> Cheers, > >> Max > >> > >> On Sun, Feb 7, 2016 at 7:12 PM, Stephan Ewen <[hidden email] > <javascript:;>> wrote: > >> > Hi! > >> > > >> > I basically test with "mvn clean install" as well. > >> > > >> > Greetings, > >> > Stephan > >> > > >> > > >> > On Sat, Feb 6, 2016 at 10:31 PM, Stefano Baghino < > >> > [hidden email] <javascript:;>> wrote: > >> > > >> >> Hi everybody, > >> >> > >> >> as I've recently started contributing to Flink, I'd like to ask you a > >> few > >> >> tips on how you normally run the tests to ensure that everything runs > >> >> smoothly before pushing the code to CI. > >> >> > >> >> First, I'd like to point out two issues in particular I had: > >> >> > >> >> 1. some test suites (KMeansWithBroadcastSetITCase.java, > >> >> ScalaCsvReaderWithPOJOITCase.scala and CsvReaderITCase.java) kept > >> >> failing > >> >> because the expected results (string literals) were matched > against > >> an > >> >> object rendered as a string using the String.format method, a > method > >> >> whose > >> >> result depends on the default Locale; as my Locale (Italian) > renders > >> >> doubles with a comma instead of a dot as the decimal separator, > the > >> >> representation of doubles diverged from the expected one, thus > >> making my > >> >> tests fail; I've solved it locally by explicitly using the US > locale > >> for > >> >> those calls — I'm thinking about opening an issue and submitting > my > >> >> solution as a PR; > >> >> 2. I had another problem with WebFrontendITCase.java: the test > suite > >> >> starts a mini cluster with the web server on port 8081; not > knowing > >> this > >> >> and (unknowingly) having another process listening on that port, > the > >> >> test > >> >> suite kept failing at initialization (line 59); however, the > message > >> >> regarding the port being busy was not displayed, despite being > >> logged at > >> >> ERROR level (WebMonitorUtils.startWebRuntimeMonitor, line 138); > what > >> >> configuration file should I edit in order for these issues to > come up > >> >> instantly without having to dig into the code to understand what's > >> >> happening? > >> >> > >> >> Apart from these issues in particular, I'd like to ask if you have > any > >> >> particular routine when it comes to building and testing Flink > locally. > >> I'm > >> >> not particularly knowledgable regarding Maven, so right now I'm > >> basically > >> >> running `mvn [clean] install [-rf :<last-failed-module>]` each time > to > >> make > >> >> sure I have a fresh build to test. Would this be the right path or is > >> there > >> >> a quicker way to have a fresh build and running the tests on them? > Feel > >> >> free to point me to any relevant documentation, if you wish. > >> >> > >> >> Thank you in advance for any tip. :) > >> >> > >> >> -- > >> >> BR, > >> >> Stefano Baghino > >> >> > >> >> Software Engineer @ Radicalbit > >> >> > >> > > > > > > > > -- > > BR, > > Stefano Baghino > > > > Software Engineer @ Radicalbit > -- BR, Stefano Baghino Software Engineer @ Radicalbit |
Free forum by Nabble | Edit this page |