Hello everyone,
I just noticed an odd behavior that occurs when using the python api start scripts: i suddenly get another argument i didn't get before; the flink-python jar path. which inadvertently breaks everything. An example as to how the execution of an example should go: bin\pyflink2.sh resources\python\flink\example\Wordcount.py (1 argument) the pyflink2.sh script then calls: bin\flink -v libraries\<python-jar> 2(this is the python version) resources\python\flink\example\Wordcount.py (3 arguments) which is supposed to pass the arguments "2" and "resources\python\flink\example\Wordcount.py" to the PythonPlanBinder's main method. Now for some reason the main method gets 3 arguments, the path to the python jar, and the 2 arguments it is supposed to get. I haven't changed the scripts in a long time, so i assume for now that an external change is the cause of this. Did anyone work on something that could cause this? |
Well, according to the git history, it was only me who touched the file
many months ago: https://github.com/apache/flink/commits/master/flink-dist/src/main/flink-bin/bin/pyflink2.sh On Wed, Jan 27, 2016 at 6:05 PM, Chesnay Schepler <[hidden email]> wrote: > Hello everyone, > > I just noticed an odd behavior that occurs when using the python api start > scripts: i suddenly get another argument i didn't get before; the > flink-python jar path. which inadvertently breaks everything. > > An example as to how the execution of an example should go: > > bin\pyflink2.sh resources\python\flink\example\Wordcount.py > (1 argument) > > the pyflink2.sh script then calls: > > bin\flink -v libraries\<python-jar> 2(this is the python version) > resources\python\flink\example\Wordcount.py > (3 arguments) > > which is supposed to pass the arguments "2" and > "resources\python\flink\example\Wordcount.py" to the PythonPlanBinder's > main method. > > Now for some reason the main method gets 3 arguments, the path to the > python jar, and the 2 arguments it is supposed to get. > > I haven't changed the scripts in a long time, so i assume for now that an > external change is the cause of this. Did anyone work on something that > could cause this? > > |
I was trending more towards changes to the CLI.
On 27.01.2016 18:48, Robert Metzger wrote: > Well, according to the git history, it was only me who touched the file > many months ago: > https://github.com/apache/flink/commits/master/flink-dist/src/main/flink-bin/bin/pyflink2.sh > > On Wed, Jan 27, 2016 at 6:05 PM, Chesnay Schepler <[hidden email]> > wrote: > >> Hello everyone, >> >> I just noticed an odd behavior that occurs when using the python api start >> scripts: i suddenly get another argument i didn't get before; the >> flink-python jar path. which inadvertently breaks everything. >> >> An example as to how the execution of an example should go: >> >> bin\pyflink2.sh resources\python\flink\example\Wordcount.py >> (1 argument) >> >> the pyflink2.sh script then calls: >> >> bin\flink -v libraries\<python-jar> 2(this is the python version) >> resources\python\flink\example\Wordcount.py >> (3 arguments) >> >> which is supposed to pass the arguments "2" and >> "resources\python\flink\example\Wordcount.py" to the PythonPlanBinder's >> main method. >> >> Now for some reason the main method gets 3 arguments, the path to the >> python jar, and the 2 arguments it is supposed to get. >> >> I haven't changed the scripts in a long time, so i assume for now that an >> external change is the cause of this. Did anyone work on something that >> could cause this? >> >> |
Given the direction of the path separators, I guess you are using Microsoft
Windows. I think Fabian is very experienced fixing windows path issues before every release ... and usually the issue are paths containing spaces. Maybe that's the case here as well? On Wed, Jan 27, 2016 at 6:51 PM, Chesnay Schepler <[hidden email]> wrote: > I was trending more towards changes to the CLI. > > > On 27.01.2016 18:48, Robert Metzger wrote: > >> Well, according to the git history, it was only me who touched the file >> many months ago: >> >> https://github.com/apache/flink/commits/master/flink-dist/src/main/flink-bin/bin/pyflink2.sh >> >> On Wed, Jan 27, 2016 at 6:05 PM, Chesnay Schepler <[hidden email]> >> wrote: >> >> Hello everyone, >>> >>> I just noticed an odd behavior that occurs when using the python api >>> start >>> scripts: i suddenly get another argument i didn't get before; the >>> flink-python jar path. which inadvertently breaks everything. >>> >>> An example as to how the execution of an example should go: >>> >>> bin\pyflink2.sh resources\python\flink\example\Wordcount.py >>> (1 argument) >>> >>> the pyflink2.sh script then calls: >>> >>> bin\flink -v libraries\<python-jar> 2(this is the python version) >>> resources\python\flink\example\Wordcount.py >>> (3 arguments) >>> >>> which is supposed to pass the arguments "2" and >>> "resources\python\flink\example\Wordcount.py" to the PythonPlanBinder's >>> main method. >>> >>> Now for some reason the main method gets 3 arguments, the path to the >>> python jar, and the 2 arguments it is supposed to get. >>> >>> I haven't changed the scripts in a long time, so i assume for now that an >>> external change is the cause of this. Did anyone work on something that >>> could cause this? >>> >>> >>> > |
Yes i am on windows. no, ain't no path containing spaces.
if it were due to spaces i would assume that either a) the python jar wouldn't be called properly or b) any of the arguments would be malformed, but they aren't. the following are the actual arguments that are passed to the PythonPlanBinder main method as printed by Arrays.toString(): [C:\Users\Zento\Documents\GitHub\flink\flink-dist\target\flink-1.0-SNAPSHOT-bin\flink-1.0-SNAPSHOT\lib\flink-python_2.10-1.0-SNAPSHOT.jar, 2, resources\python\flink\example\WordCount.py] Supposedly, these are the exact arguments passed to bin\flink . i could just ignore the additional argument and have it work again, but i would still like to know why that argument even arrives here. On 27.01.2016 18:58, Robert Metzger wrote: > Given the direction of the path separators, I guess you are using Microsoft > Windows. I think Fabian is very experienced fixing windows path issues > before every release ... and usually the issue are paths containing spaces. > Maybe that's the case here as well? > > On Wed, Jan 27, 2016 at 6:51 PM, Chesnay Schepler <[hidden email]> > wrote: > >> I was trending more towards changes to the CLI. >> >> >> On 27.01.2016 18:48, Robert Metzger wrote: >> >>> Well, according to the git history, it was only me who touched the file >>> many months ago: >>> >>> https://github.com/apache/flink/commits/master/flink-dist/src/main/flink-bin/bin/pyflink2.sh >>> >>> On Wed, Jan 27, 2016 at 6:05 PM, Chesnay Schepler <[hidden email]> >>> wrote: >>> >>> Hello everyone, >>>> I just noticed an odd behavior that occurs when using the python api >>>> start >>>> scripts: i suddenly get another argument i didn't get before; the >>>> flink-python jar path. which inadvertently breaks everything. >>>> >>>> An example as to how the execution of an example should go: >>>> >>>> bin\pyflink2.sh resources\python\flink\example\Wordcount.py >>>> (1 argument) >>>> >>>> the pyflink2.sh script then calls: >>>> >>>> bin\flink -v libraries\<python-jar> 2(this is the python version) >>>> resources\python\flink\example\Wordcount.py >>>> (3 arguments) >>>> >>>> which is supposed to pass the arguments "2" and >>>> "resources\python\flink\example\Wordcount.py" to the PythonPlanBinder's >>>> main method. >>>> >>>> Now for some reason the main method gets 3 arguments, the path to the >>>> python jar, and the 2 arguments it is supposed to get. >>>> >>>> I haven't changed the scripts in a long time, so i assume for now that an >>>> external change is the cause of this. Did anyone work on something that >>>> could cause this? >>>> >>>> >>>> |
i went back 30 commits and the issue is gone, as such its not a system
problem. gonna narrow it down now. On 27.01.2016 19:29, Chesnay Schepler wrote: > Yes i am on windows. no, ain't no path containing spaces. > > if it were due to spaces i would assume that either a) the python jar > wouldn't be called properly or b) any of the arguments would be > malformed, but they aren't. > > the following are the actual arguments that are passed to the > PythonPlanBinder main method as printed by Arrays.toString(): > > [C:\Users\Zento\Documents\GitHub\flink\flink-dist\target\flink-1.0-SNAPSHOT-bin\flink-1.0-SNAPSHOT\lib\flink-python_2.10-1.0-SNAPSHOT.jar, > 2, resources\python\flink\example\WordCount.py] > > Supposedly, these are the exact arguments passed to bin\flink . > > i could just ignore the additional argument and have it work again, > but i would still like to know why that argument even arrives here. > > On 27.01.2016 18:58, Robert Metzger wrote: >> Given the direction of the path separators, I guess you are using >> Microsoft >> Windows. I think Fabian is very experienced fixing windows path issues >> before every release ... and usually the issue are paths containing >> spaces. >> Maybe that's the case here as well? >> >> On Wed, Jan 27, 2016 at 6:51 PM, Chesnay Schepler <[hidden email]> >> wrote: >> >>> I was trending more towards changes to the CLI. >>> >>> >>> On 27.01.2016 18:48, Robert Metzger wrote: >>> >>>> Well, according to the git history, it was only me who touched the >>>> file >>>> many months ago: >>>> >>>> https://github.com/apache/flink/commits/master/flink-dist/src/main/flink-bin/bin/pyflink2.sh >>>> >>>> >>>> On Wed, Jan 27, 2016 at 6:05 PM, Chesnay Schepler <[hidden email]> >>>> wrote: >>>> >>>> Hello everyone, >>>>> I just noticed an odd behavior that occurs when using the python api >>>>> start >>>>> scripts: i suddenly get another argument i didn't get before; the >>>>> flink-python jar path. which inadvertently breaks everything. >>>>> >>>>> An example as to how the execution of an example should go: >>>>> >>>>> bin\pyflink2.sh resources\python\flink\example\Wordcount.py >>>>> (1 argument) >>>>> >>>>> the pyflink2.sh script then calls: >>>>> >>>>> bin\flink -v libraries\<python-jar> 2(this is the python version) >>>>> resources\python\flink\example\Wordcount.py >>>>> (3 arguments) >>>>> >>>>> which is supposed to pass the arguments "2" and >>>>> "resources\python\flink\example\Wordcount.py" to the >>>>> PythonPlanBinder's >>>>> main method. >>>>> >>>>> Now for some reason the main method gets 3 arguments, the path to the >>>>> python jar, and the 2 arguments it is supposed to get. >>>>> >>>>> I haven't changed the scripts in a long time, so i assume for now >>>>> that an >>>>> external change is the cause of this. Did anyone work on something >>>>> that >>>>> could cause this? >>>>> >>>>> >>>>> > > |
Running mvn clean package resolved the issue. still the weirdest thing
that happened this week. :/ On 27.01.2016 20:19, Chesnay Schepler wrote: > i went back 30 commits and the issue is gone, as such its not a system > problem. gonna narrow it down now. > > On 27.01.2016 19:29, Chesnay Schepler wrote: >> Yes i am on windows. no, ain't no path containing spaces. >> >> if it were due to spaces i would assume that either a) the python jar >> wouldn't be called properly or b) any of the arguments would be >> malformed, but they aren't. >> >> the following are the actual arguments that are passed to the >> PythonPlanBinder main method as printed by Arrays.toString(): >> >> [C:\Users\Zento\Documents\GitHub\flink\flink-dist\target\flink-1.0-SNAPSHOT-bin\flink-1.0-SNAPSHOT\lib\flink-python_2.10-1.0-SNAPSHOT.jar, >> 2, resources\python\flink\example\WordCount.py] >> >> Supposedly, these are the exact arguments passed to bin\flink . >> >> i could just ignore the additional argument and have it work again, >> but i would still like to know why that argument even arrives here. >> >> On 27.01.2016 18:58, Robert Metzger wrote: >>> Given the direction of the path separators, I guess you are using >>> Microsoft >>> Windows. I think Fabian is very experienced fixing windows path issues >>> before every release ... and usually the issue are paths containing >>> spaces. >>> Maybe that's the case here as well? >>> >>> On Wed, Jan 27, 2016 at 6:51 PM, Chesnay Schepler <[hidden email]> >>> wrote: >>> >>>> I was trending more towards changes to the CLI. >>>> >>>> >>>> On 27.01.2016 18:48, Robert Metzger wrote: >>>> >>>>> Well, according to the git history, it was only me who touched the >>>>> file >>>>> many months ago: >>>>> >>>>> https://github.com/apache/flink/commits/master/flink-dist/src/main/flink-bin/bin/pyflink2.sh >>>>> >>>>> >>>>> On Wed, Jan 27, 2016 at 6:05 PM, Chesnay Schepler >>>>> <[hidden email]> >>>>> wrote: >>>>> >>>>> Hello everyone, >>>>>> I just noticed an odd behavior that occurs when using the python api >>>>>> start >>>>>> scripts: i suddenly get another argument i didn't get before; the >>>>>> flink-python jar path. which inadvertently breaks everything. >>>>>> >>>>>> An example as to how the execution of an example should go: >>>>>> >>>>>> bin\pyflink2.sh resources\python\flink\example\Wordcount.py >>>>>> (1 argument) >>>>>> >>>>>> the pyflink2.sh script then calls: >>>>>> >>>>>> bin\flink -v libraries\<python-jar> 2(this is the python version) >>>>>> resources\python\flink\example\Wordcount.py >>>>>> (3 arguments) >>>>>> >>>>>> which is supposed to pass the arguments "2" and >>>>>> "resources\python\flink\example\Wordcount.py" to the >>>>>> PythonPlanBinder's >>>>>> main method. >>>>>> >>>>>> Now for some reason the main method gets 3 arguments, the path to >>>>>> the >>>>>> python jar, and the 2 arguments it is supposed to get. >>>>>> >>>>>> I haven't changed the scripts in a long time, so i assume for now >>>>>> that an >>>>>> external change is the cause of this. Did anyone work on >>>>>> something that >>>>>> could cause this? >>>>>> >>>>>> >>>>>> >> >> > > |
Free forum by Nabble | Edit this page |