Hi Flink community,
while reviewing the LICENSE and NOTICE file of Apache Flink, I noticed that according to the LICENSE file Flink contains many java script files. However, tracking the corresponding files back was not so easy, because they are actually all merged into flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a large java script file which is part of the result of the bower build process. I was wondering why we added a "binary" file which is auto-generated to our code base. Was/Is there any specific reason for it? The problem is that the java script files contained in the vendor.js are not exactly the set of declared bower dependencies in flink-runtime-web/web-dashboard/bower.json. I suspect that also transitive dependencies will be included. At least that is my only explanation why we're referencing lodash.js, for example, in our LICENSE file (you can find it deeply hidden in the vendor.js file). Wouldn't it be easier to auto-generate the web-dashboard related files while building Flink? This would not only clean up our repository but also make the traceability of contained 3rd party code in our code base much easier. Maybe this maven plugin [1] could help us. [1] https://github.com/eirslett/frontend-maven-plugin Cheers, Till |
Hi Till,
Thanks for checking the licenses for our web frontend. I think the reason why we added a big binary Javascript blob into our repository was that it was easiest for most developers to deal with. We don't have much Javascript expertise in the Flink community. Incorporating the web frontend in the build process would require some additional work and make it even more complicated. It this sense the solution we have currently is quite nice because it doesn't require developers to fiddle around with Javascript as long as they are not working on the webfrontend. If they do, they can simply use NPM and Bower which install the listed dependencies. The disadvantage is a slight increase of our repository because we commit a new "vendor.js" for every recompile, but that is negligible in my opinion. It would be cleaner to build the Javascript parts in the Maven build process but it will complicate the build process further. Frankly, I don't see it'll add much value. Considering the fragility of our build system I might be a bit conservative here :) Cheers, Max On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <[hidden email]> wrote: > Hi Flink community, > > while reviewing the LICENSE and NOTICE file of Apache Flink, I noticed that > according to the LICENSE file Flink contains many java script files. > However, tracking the corresponding files back was not so easy, because > they are actually all merged into > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a large java > script file which is part of the result of the bower build process. > > I was wondering why we added a "binary" file which is auto-generated to our > code base. Was/Is there any specific reason for it? > > The problem is that the java script files contained in the vendor.js are > not exactly the set of declared bower dependencies in > flink-runtime-web/web-dashboard/bower.json. I suspect that also transitive > dependencies will be included. At least that is my only explanation why > we're referencing lodash.js, for example, in our LICENSE file (you can find > it deeply hidden in the vendor.js file). > > Wouldn't it be easier to auto-generate the web-dashboard related files > while building Flink? This would not only clean up our repository but also > make the traceability of contained 3rd party code in our code base much > easier. Maybe this maven plugin [1] could help us. > > [1] https://github.com/eirslett/frontend-maven-plugin > > Cheers, > Till |
I think it's not a question of easy-of-use but one of licensing. I don't
think anyone really knows what code ends up in vendor.js, so it is very hard to figure out what we have to put into our LICENSE file. On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <[hidden email]> wrote: > Hi Till, > > Thanks for checking the licenses for our web frontend. > > I think the reason why we added a big binary Javascript blob into our > repository was that it was easiest for most developers to deal with. > We don't have much Javascript expertise in the Flink community. > Incorporating the web frontend in the build process would require some > additional work and make it even more complicated. It this sense the > solution we have currently is quite nice because it doesn't require > developers to fiddle around with Javascript as long as they are not > working on the webfrontend. If they do, they can simply use NPM and > Bower which install the listed dependencies. The disadvantage is a > slight increase of our repository because we commit a new "vendor.js" > for every recompile, but that is negligible in my opinion. > > It would be cleaner to build the Javascript parts in the Maven build > process but it will complicate the build process further. Frankly, I > don't see it'll add much value. Considering the fragility of our build > system I might be a bit conservative here :) > > Cheers, > Max > > On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <[hidden email]> > wrote: > > Hi Flink community, > > > > while reviewing the LICENSE and NOTICE file of Apache Flink, I noticed > that > > according to the LICENSE file Flink contains many java script files. > > However, tracking the corresponding files back was not so easy, because > > they are actually all merged into > > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a large > java > > script file which is part of the result of the bower build process. > > > > I was wondering why we added a "binary" file which is auto-generated to > our > > code base. Was/Is there any specific reason for it? > > > > The problem is that the java script files contained in the vendor.js are > > not exactly the set of declared bower dependencies in > > flink-runtime-web/web-dashboard/bower.json. I suspect that also > transitive > > dependencies will be included. At least that is my only explanation why > > we're referencing lodash.js, for example, in our LICENSE file (you can > find > > it deeply hidden in the vendor.js file). > > > > Wouldn't it be easier to auto-generate the web-dashboard related files > > while building Flink? This would not only clean up our repository but > also > > make the traceability of contained 3rd party code in our code base much > > easier. Maybe this maven plugin [1] could help us. > > > > [1] https://github.com/eirslett/frontend-maven-plugin > > > > Cheers, > > Till > |
I'm afraid I don't think integration in the Maven build process makes
a difference in terms of licensing. It is already clearly specified what dependencies the web frontend uses (see README, bower.json, package.json). It won't get any easier with something integrated in the build process. We still have to leverage the Javascript build systems which resolve transitive dependencies in the same way Maven does. In the end, we just have to properly check out licenses which takes time. In what sense do you think it would make licensing easier? On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek <[hidden email]> wrote: > I think it's not a question of easy-of-use but one of licensing. I don't > think anyone really knows what code ends up in vendor.js, so it is very > hard to figure out what we have to put into our LICENSE file. > > On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <[hidden email]> wrote: > >> Hi Till, >> >> Thanks for checking the licenses for our web frontend. >> >> I think the reason why we added a big binary Javascript blob into our >> repository was that it was easiest for most developers to deal with. >> We don't have much Javascript expertise in the Flink community. >> Incorporating the web frontend in the build process would require some >> additional work and make it even more complicated. It this sense the >> solution we have currently is quite nice because it doesn't require >> developers to fiddle around with Javascript as long as they are not >> working on the webfrontend. If they do, they can simply use NPM and >> Bower which install the listed dependencies. The disadvantage is a >> slight increase of our repository because we commit a new "vendor.js" >> for every recompile, but that is negligible in my opinion. >> >> It would be cleaner to build the Javascript parts in the Maven build >> process but it will complicate the build process further. Frankly, I >> don't see it'll add much value. Considering the fragility of our build >> system I might be a bit conservative here :) >> >> Cheers, >> Max >> >> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <[hidden email]> >> wrote: >> > Hi Flink community, >> > >> > while reviewing the LICENSE and NOTICE file of Apache Flink, I noticed >> that >> > according to the LICENSE file Flink contains many java script files. >> > However, tracking the corresponding files back was not so easy, because >> > they are actually all merged into >> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a large >> java >> > script file which is part of the result of the bower build process. >> > >> > I was wondering why we added a "binary" file which is auto-generated to >> our >> > code base. Was/Is there any specific reason for it? >> > >> > The problem is that the java script files contained in the vendor.js are >> > not exactly the set of declared bower dependencies in >> > flink-runtime-web/web-dashboard/bower.json. I suspect that also >> transitive >> > dependencies will be included. At least that is my only explanation why >> > we're referencing lodash.js, for example, in our LICENSE file (you can >> find >> > it deeply hidden in the vendor.js file). >> > >> > Wouldn't it be easier to auto-generate the web-dashboard related files >> > while building Flink? This would not only clean up our repository but >> also >> > make the traceability of contained 3rd party code in our code base much >> > easier. Maybe this maven plugin [1] could help us. >> > >> > [1] https://github.com/eirslett/frontend-maven-plugin >> > >> > Cheers, >> > Till >> |
AFAIK the Apache License requires that we have an entry in the
LICENSE/NOTICE file for all external code that we ship with our source. If we have vendor.js in our source we have to include everything that is in there. If we don't have any actual external code in our source but only specified dependencies we should not be required to include them in the LICENSE/NOTICE file. On Thu, 30 Jun 2016 at 14:07 Maximilian Michels <[hidden email]> wrote: > I'm afraid I don't think integration in the Maven build process makes > a difference in terms of licensing. It is already clearly specified > what dependencies the web frontend uses (see README, bower.json, > package.json). It won't get any easier with something integrated in > the build process. We still have to leverage the Javascript build > systems which resolve transitive dependencies in the same way Maven > does. In the end, we just have to properly check out licenses which > takes time. > > In what sense do you think it would make licensing easier? > > On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek <[hidden email]> > wrote: > > I think it's not a question of easy-of-use but one of licensing. I don't > > think anyone really knows what code ends up in vendor.js, so it is very > > hard to figure out what we have to put into our LICENSE file. > > > > On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <[hidden email]> wrote: > > > >> Hi Till, > >> > >> Thanks for checking the licenses for our web frontend. > >> > >> I think the reason why we added a big binary Javascript blob into our > >> repository was that it was easiest for most developers to deal with. > >> We don't have much Javascript expertise in the Flink community. > >> Incorporating the web frontend in the build process would require some > >> additional work and make it even more complicated. It this sense the > >> solution we have currently is quite nice because it doesn't require > >> developers to fiddle around with Javascript as long as they are not > >> working on the webfrontend. If they do, they can simply use NPM and > >> Bower which install the listed dependencies. The disadvantage is a > >> slight increase of our repository because we commit a new "vendor.js" > >> for every recompile, but that is negligible in my opinion. > >> > >> It would be cleaner to build the Javascript parts in the Maven build > >> process but it will complicate the build process further. Frankly, I > >> don't see it'll add much value. Considering the fragility of our build > >> system I might be a bit conservative here :) > >> > >> Cheers, > >> Max > >> > >> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <[hidden email]> > >> wrote: > >> > Hi Flink community, > >> > > >> > while reviewing the LICENSE and NOTICE file of Apache Flink, I noticed > >> that > >> > according to the LICENSE file Flink contains many java script files. > >> > However, tracking the corresponding files back was not so easy, > because > >> > they are actually all merged into > >> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a large > >> java > >> > script file which is part of the result of the bower build process. > >> > > >> > I was wondering why we added a "binary" file which is auto-generated > to > >> our > >> > code base. Was/Is there any specific reason for it? > >> > > >> > The problem is that the java script files contained in the vendor.js > are > >> > not exactly the set of declared bower dependencies in > >> > flink-runtime-web/web-dashboard/bower.json. I suspect that also > >> transitive > >> > dependencies will be included. At least that is my only explanation > why > >> > we're referencing lodash.js, for example, in our LICENSE file (you can > >> find > >> > it deeply hidden in the vendor.js file). > >> > > >> > Wouldn't it be easier to auto-generate the web-dashboard related files > >> > while building Flink? This would not only clean up our repository but > >> also > >> > make the traceability of contained 3rd party code in our code base > much > >> > easier. Maybe this maven plugin [1] could help us. > >> > > >> > [1] https://github.com/eirslett/frontend-maven-plugin > >> > > >> > Cheers, > >> > Till > >> > |
Yes, that is also how I understood the Apache License requirements. Having
a mere dependency on some external library which is not shipped as part of the source release does not require to include it in the LICENSE/NOTICE file. I think that you only have to include a reference in the LICENSE/NOTICE file, if your source release contains actual code which has a copyright. With the vendor.js file, the latter case applies. I'm not a bower expert, so I cannot tell for sure which javascript files end up in the vendor.js file. I would assume that they are the bower dependencies defined in bower.json and their transitive dependencies. But parsing the vendor.js file will not be feasible, since it contains 80000+ LOC. Thus my main concern, why I initiated this discussion, is the legal aspect of including a monolithic file of javascript code in our code base. The Flink community might release code which violates other people's copyright. Additionally, it might be worth the effort to harden our build system anyway if you say that it's fragile. Then we could also properly integrate the web-dashboard generation into the build process. On Thu, Jun 30, 2016 at 2:30 PM, Aljoscha Krettek <[hidden email]> wrote: > AFAIK the Apache License requires that we have an entry in the > LICENSE/NOTICE file for all external code that we ship with our source. If > we have vendor.js in our source we have to include everything that is in > there. If we don't have any actual external code in our source but only > specified dependencies we should not be required to include them in the > LICENSE/NOTICE file. > > On Thu, 30 Jun 2016 at 14:07 Maximilian Michels <[hidden email]> wrote: > > > I'm afraid I don't think integration in the Maven build process makes > > a difference in terms of licensing. It is already clearly specified > > what dependencies the web frontend uses (see README, bower.json, > > package.json). It won't get any easier with something integrated in > > the build process. We still have to leverage the Javascript build > > systems which resolve transitive dependencies in the same way Maven > > does. In the end, we just have to properly check out licenses which > > takes time. > > > > In what sense do you think it would make licensing easier? > > > > On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek <[hidden email]> > > wrote: > > > I think it's not a question of easy-of-use but one of licensing. I > don't > > > think anyone really knows what code ends up in vendor.js, so it is very > > > hard to figure out what we have to put into our LICENSE file. > > > > > > On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <[hidden email]> > wrote: > > > > > >> Hi Till, > > >> > > >> Thanks for checking the licenses for our web frontend. > > >> > > >> I think the reason why we added a big binary Javascript blob into our > > >> repository was that it was easiest for most developers to deal with. > > >> We don't have much Javascript expertise in the Flink community. > > >> Incorporating the web frontend in the build process would require some > > >> additional work and make it even more complicated. It this sense the > > >> solution we have currently is quite nice because it doesn't require > > >> developers to fiddle around with Javascript as long as they are not > > >> working on the webfrontend. If they do, they can simply use NPM and > > >> Bower which install the listed dependencies. The disadvantage is a > > >> slight increase of our repository because we commit a new "vendor.js" > > >> for every recompile, but that is negligible in my opinion. > > >> > > >> It would be cleaner to build the Javascript parts in the Maven build > > >> process but it will complicate the build process further. Frankly, I > > >> don't see it'll add much value. Considering the fragility of our build > > >> system I might be a bit conservative here :) > > >> > > >> Cheers, > > >> Max > > >> > > >> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <[hidden email]> > > >> wrote: > > >> > Hi Flink community, > > >> > > > >> > while reviewing the LICENSE and NOTICE file of Apache Flink, I > noticed > > >> that > > >> > according to the LICENSE file Flink contains many java script files. > > >> > However, tracking the corresponding files back was not so easy, > > because > > >> > they are actually all merged into > > >> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a > large > > >> java > > >> > script file which is part of the result of the bower build process. > > >> > > > >> > I was wondering why we added a "binary" file which is auto-generated > > to > > >> our > > >> > code base. Was/Is there any specific reason for it? > > >> > > > >> > The problem is that the java script files contained in the vendor.js > > are > > >> > not exactly the set of declared bower dependencies in > > >> > flink-runtime-web/web-dashboard/bower.json. I suspect that also > > >> transitive > > >> > dependencies will be included. At least that is my only explanation > > why > > >> > we're referencing lodash.js, for example, in our LICENSE file (you > can > > >> find > > >> > it deeply hidden in the vendor.js file). > > >> > > > >> > Wouldn't it be easier to auto-generate the web-dashboard related > files > > >> > while building Flink? This would not only clean up our repository > but > > >> also > > >> > make the traceability of contained 3rd party code in our code base > > much > > >> > easier. Maybe this maven plugin [1] could help us. > > >> > > > >> > [1] https://github.com/eirslett/frontend-maven-plugin > > >> > > > >> > Cheers, > > >> > Till > > >> > > > |
Ah, now I see that a compile-time dependency resolution would make
sense. Then we don't have to check license compatibility for web dependencies which are downloaded during compile time and are not part of the source distribution. +1 Would be worth the effort to integrate this in our build system, e.g. via the proposed plugin. On Thu, Jun 30, 2016 at 2:48 PM, Till Rohrmann <[hidden email]> wrote: > Yes, that is also how I understood the Apache License requirements. Having > a mere dependency on some external library which is not shipped as part of > the source release does not require to include it in the LICENSE/NOTICE > file. I think that you only have to include a reference in the > LICENSE/NOTICE file, if your source release contains actual code which has > a copyright. With the vendor.js file, the latter case applies. > > I'm not a bower expert, so I cannot tell for sure which javascript files > end up in the vendor.js file. I would assume that they are the bower > dependencies defined in bower.json and their transitive dependencies. But > parsing the vendor.js file will not be feasible, since it contains 80000+ > LOC. > > Thus my main concern, why I initiated this discussion, is the legal aspect > of including a monolithic file of javascript code in our code base. The > Flink community might release code which violates other people's copyright. > > Additionally, it might be worth the effort to harden our build system > anyway if you say that it's fragile. Then we could also properly integrate > the web-dashboard generation into the build process. > > On Thu, Jun 30, 2016 at 2:30 PM, Aljoscha Krettek <[hidden email]> > wrote: > >> AFAIK the Apache License requires that we have an entry in the >> LICENSE/NOTICE file for all external code that we ship with our source. If >> we have vendor.js in our source we have to include everything that is in >> there. If we don't have any actual external code in our source but only >> specified dependencies we should not be required to include them in the >> LICENSE/NOTICE file. >> >> On Thu, 30 Jun 2016 at 14:07 Maximilian Michels <[hidden email]> wrote: >> >> > I'm afraid I don't think integration in the Maven build process makes >> > a difference in terms of licensing. It is already clearly specified >> > what dependencies the web frontend uses (see README, bower.json, >> > package.json). It won't get any easier with something integrated in >> > the build process. We still have to leverage the Javascript build >> > systems which resolve transitive dependencies in the same way Maven >> > does. In the end, we just have to properly check out licenses which >> > takes time. >> > >> > In what sense do you think it would make licensing easier? >> > >> > On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek <[hidden email]> >> > wrote: >> > > I think it's not a question of easy-of-use but one of licensing. I >> don't >> > > think anyone really knows what code ends up in vendor.js, so it is very >> > > hard to figure out what we have to put into our LICENSE file. >> > > >> > > On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <[hidden email]> >> wrote: >> > > >> > >> Hi Till, >> > >> >> > >> Thanks for checking the licenses for our web frontend. >> > >> >> > >> I think the reason why we added a big binary Javascript blob into our >> > >> repository was that it was easiest for most developers to deal with. >> > >> We don't have much Javascript expertise in the Flink community. >> > >> Incorporating the web frontend in the build process would require some >> > >> additional work and make it even more complicated. It this sense the >> > >> solution we have currently is quite nice because it doesn't require >> > >> developers to fiddle around with Javascript as long as they are not >> > >> working on the webfrontend. If they do, they can simply use NPM and >> > >> Bower which install the listed dependencies. The disadvantage is a >> > >> slight increase of our repository because we commit a new "vendor.js" >> > >> for every recompile, but that is negligible in my opinion. >> > >> >> > >> It would be cleaner to build the Javascript parts in the Maven build >> > >> process but it will complicate the build process further. Frankly, I >> > >> don't see it'll add much value. Considering the fragility of our build >> > >> system I might be a bit conservative here :) >> > >> >> > >> Cheers, >> > >> Max >> > >> >> > >> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann <[hidden email]> >> > >> wrote: >> > >> > Hi Flink community, >> > >> > >> > >> > while reviewing the LICENSE and NOTICE file of Apache Flink, I >> noticed >> > >> that >> > >> > according to the LICENSE file Flink contains many java script files. >> > >> > However, tracking the corresponding files back was not so easy, >> > because >> > >> > they are actually all merged into >> > >> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a >> large >> > >> java >> > >> > script file which is part of the result of the bower build process. >> > >> > >> > >> > I was wondering why we added a "binary" file which is auto-generated >> > to >> > >> our >> > >> > code base. Was/Is there any specific reason for it? >> > >> > >> > >> > The problem is that the java script files contained in the vendor.js >> > are >> > >> > not exactly the set of declared bower dependencies in >> > >> > flink-runtime-web/web-dashboard/bower.json. I suspect that also >> > >> transitive >> > >> > dependencies will be included. At least that is my only explanation >> > why >> > >> > we're referencing lodash.js, for example, in our LICENSE file (you >> can >> > >> find >> > >> > it deeply hidden in the vendor.js file). >> > >> > >> > >> > Wouldn't it be easier to auto-generate the web-dashboard related >> files >> > >> > while building Flink? This would not only clean up our repository >> but >> > >> also >> > >> > make the traceability of contained 3rd party code in our code base >> > much >> > >> > easier. Maybe this maven plugin [1] could help us. >> > >> > >> > >> > [1] https://github.com/eirslett/frontend-maven-plugin >> > >> > >> > >> > Cheers, >> > >> > Till >> > >> >> > >> |
These external libraries will still be shipped in the binaries, which
complicates the issue since source and binary licenses must be tracked separately. http://www.apache.org/dev/licensing-howto.html#binary I do agree that this would be a nice improvement to the build system. On Fri, Jul 1, 2016 at 11:16 AM, Maximilian Michels <[hidden email]> wrote: > Ah, now I see that a compile-time dependency resolution would make > sense. Then we don't have to check license compatibility for web > dependencies which are downloaded during compile time and are not part > of the source distribution. > > +1 Would be worth the effort to integrate this in our build system, > e.g. via the proposed plugin. > > On Thu, Jun 30, 2016 at 2:48 PM, Till Rohrmann <[hidden email]> > wrote: > > Yes, that is also how I understood the Apache License requirements. > Having > > a mere dependency on some external library which is not shipped as part > of > > the source release does not require to include it in the LICENSE/NOTICE > > file. I think that you only have to include a reference in the > > LICENSE/NOTICE file, if your source release contains actual code which > has > > a copyright. With the vendor.js file, the latter case applies. > > > > I'm not a bower expert, so I cannot tell for sure which javascript files > > end up in the vendor.js file. I would assume that they are the bower > > dependencies defined in bower.json and their transitive dependencies. But > > parsing the vendor.js file will not be feasible, since it contains 80000+ > > LOC. > > > > Thus my main concern, why I initiated this discussion, is the legal > aspect > > of including a monolithic file of javascript code in our code base. The > > Flink community might release code which violates other people's > copyright. > > > > Additionally, it might be worth the effort to harden our build system > > anyway if you say that it's fragile. Then we could also properly > integrate > > the web-dashboard generation into the build process. > > > > On Thu, Jun 30, 2016 at 2:30 PM, Aljoscha Krettek <[hidden email]> > > wrote: > > > >> AFAIK the Apache License requires that we have an entry in the > >> LICENSE/NOTICE file for all external code that we ship with our source. > If > >> we have vendor.js in our source we have to include everything that is in > >> there. If we don't have any actual external code in our source but only > >> specified dependencies we should not be required to include them in the > >> LICENSE/NOTICE file. > >> > >> On Thu, 30 Jun 2016 at 14:07 Maximilian Michels <[hidden email]> wrote: > >> > >> > I'm afraid I don't think integration in the Maven build process makes > >> > a difference in terms of licensing. It is already clearly specified > >> > what dependencies the web frontend uses (see README, bower.json, > >> > package.json). It won't get any easier with something integrated in > >> > the build process. We still have to leverage the Javascript build > >> > systems which resolve transitive dependencies in the same way Maven > >> > does. In the end, we just have to properly check out licenses which > >> > takes time. > >> > > >> > In what sense do you think it would make licensing easier? > >> > > >> > On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek < > [hidden email]> > >> > wrote: > >> > > I think it's not a question of easy-of-use but one of licensing. I > >> don't > >> > > think anyone really knows what code ends up in vendor.js, so it is > very > >> > > hard to figure out what we have to put into our LICENSE file. > >> > > > >> > > On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <[hidden email]> > >> wrote: > >> > > > >> > >> Hi Till, > >> > >> > >> > >> Thanks for checking the licenses for our web frontend. > >> > >> > >> > >> I think the reason why we added a big binary Javascript blob into > our > >> > >> repository was that it was easiest for most developers to deal > with. > >> > >> We don't have much Javascript expertise in the Flink community. > >> > >> Incorporating the web frontend in the build process would require > some > >> > >> additional work and make it even more complicated. It this sense > the > >> > >> solution we have currently is quite nice because it doesn't require > >> > >> developers to fiddle around with Javascript as long as they are not > >> > >> working on the webfrontend. If they do, they can simply use NPM and > >> > >> Bower which install the listed dependencies. The disadvantage is a > >> > >> slight increase of our repository because we commit a new > "vendor.js" > >> > >> for every recompile, but that is negligible in my opinion. > >> > >> > >> > >> It would be cleaner to build the Javascript parts in the Maven > build > >> > >> process but it will complicate the build process further. Frankly, > I > >> > >> don't see it'll add much value. Considering the fragility of our > build > >> > >> system I might be a bit conservative here :) > >> > >> > >> > >> Cheers, > >> > >> Max > >> > >> > >> > >> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann < > [hidden email]> > >> > >> wrote: > >> > >> > Hi Flink community, > >> > >> > > >> > >> > while reviewing the LICENSE and NOTICE file of Apache Flink, I > >> noticed > >> > >> that > >> > >> > according to the LICENSE file Flink contains many java script > files. > >> > >> > However, tracking the corresponding files back was not so easy, > >> > because > >> > >> > they are actually all merged into > >> > >> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is a > >> large > >> > >> java > >> > >> > script file which is part of the result of the bower build > process. > >> > >> > > >> > >> > I was wondering why we added a "binary" file which is > auto-generated > >> > to > >> > >> our > >> > >> > code base. Was/Is there any specific reason for it? > >> > >> > > >> > >> > The problem is that the java script files contained in the > vendor.js > >> > are > >> > >> > not exactly the set of declared bower dependencies in > >> > >> > flink-runtime-web/web-dashboard/bower.json. I suspect that also > >> > >> transitive > >> > >> > dependencies will be included. At least that is my only > explanation > >> > why > >> > >> > we're referencing lodash.js, for example, in our LICENSE file > (you > >> can > >> > >> find > >> > >> > it deeply hidden in the vendor.js file). > >> > >> > > >> > >> > Wouldn't it be easier to auto-generate the web-dashboard related > >> files > >> > >> > while building Flink? This would not only clean up our repository > >> but > >> > >> also > >> > >> > make the traceability of contained 3rd party code in our code > base > >> > much > >> > >> > easier. Maybe this maven plugin [1] could help us. > >> > >> > > >> > >> > [1] https://github.com/eirslett/frontend-maven-plugin > >> > >> > > >> > >> > Cheers, > >> > >> > Till > >> > >> > >> > > >> > |
Hmm, this sounds like we should also have a proper LICENSE/NOTICE for our
binary releases. On Fri, 1 Jul 2016 at 18:06 Greg Hogan <[hidden email]> wrote: > These external libraries will still be shipped in the binaries, which > complicates the issue since source and binary licenses must be tracked > separately. > http://www.apache.org/dev/licensing-howto.html#binary > > I do agree that this would be a nice improvement to the build system. > > On Fri, Jul 1, 2016 at 11:16 AM, Maximilian Michels <[hidden email]> > wrote: > > > Ah, now I see that a compile-time dependency resolution would make > > sense. Then we don't have to check license compatibility for web > > dependencies which are downloaded during compile time and are not part > > of the source distribution. > > > > +1 Would be worth the effort to integrate this in our build system, > > e.g. via the proposed plugin. > > > > On Thu, Jun 30, 2016 at 2:48 PM, Till Rohrmann <[hidden email]> > > wrote: > > > Yes, that is also how I understood the Apache License requirements. > > Having > > > a mere dependency on some external library which is not shipped as part > > of > > > the source release does not require to include it in the LICENSE/NOTICE > > > file. I think that you only have to include a reference in the > > > LICENSE/NOTICE file, if your source release contains actual code which > > has > > > a copyright. With the vendor.js file, the latter case applies. > > > > > > I'm not a bower expert, so I cannot tell for sure which javascript > files > > > end up in the vendor.js file. I would assume that they are the bower > > > dependencies defined in bower.json and their transitive dependencies. > But > > > parsing the vendor.js file will not be feasible, since it contains > 80000+ > > > LOC. > > > > > > Thus my main concern, why I initiated this discussion, is the legal > > aspect > > > of including a monolithic file of javascript code in our code base. The > > > Flink community might release code which violates other people's > > copyright. > > > > > > Additionally, it might be worth the effort to harden our build system > > > anyway if you say that it's fragile. Then we could also properly > > integrate > > > the web-dashboard generation into the build process. > > > > > > On Thu, Jun 30, 2016 at 2:30 PM, Aljoscha Krettek <[hidden email] > > > > > wrote: > > > > > >> AFAIK the Apache License requires that we have an entry in the > > >> LICENSE/NOTICE file for all external code that we ship with our > source. > > If > > >> we have vendor.js in our source we have to include everything that is > in > > >> there. If we don't have any actual external code in our source but > only > > >> specified dependencies we should not be required to include them in > the > > >> LICENSE/NOTICE file. > > >> > > >> On Thu, 30 Jun 2016 at 14:07 Maximilian Michels <[hidden email]> > wrote: > > >> > > >> > I'm afraid I don't think integration in the Maven build process > makes > > >> > a difference in terms of licensing. It is already clearly specified > > >> > what dependencies the web frontend uses (see README, bower.json, > > >> > package.json). It won't get any easier with something integrated in > > >> > the build process. We still have to leverage the Javascript build > > >> > systems which resolve transitive dependencies in the same way Maven > > >> > does. In the end, we just have to properly check out licenses which > > >> > takes time. > > >> > > > >> > In what sense do you think it would make licensing easier? > > >> > > > >> > On Thu, Jun 30, 2016 at 1:47 PM, Aljoscha Krettek < > > [hidden email]> > > >> > wrote: > > >> > > I think it's not a question of easy-of-use but one of licensing. I > > >> don't > > >> > > think anyone really knows what code ends up in vendor.js, so it is > > very > > >> > > hard to figure out what we have to put into our LICENSE file. > > >> > > > > >> > > On Thu, 30 Jun 2016 at 12:14 Maximilian Michels <[hidden email]> > > >> wrote: > > >> > > > > >> > >> Hi Till, > > >> > >> > > >> > >> Thanks for checking the licenses for our web frontend. > > >> > >> > > >> > >> I think the reason why we added a big binary Javascript blob into > > our > > >> > >> repository was that it was easiest for most developers to deal > > with. > > >> > >> We don't have much Javascript expertise in the Flink community. > > >> > >> Incorporating the web frontend in the build process would require > > some > > >> > >> additional work and make it even more complicated. It this sense > > the > > >> > >> solution we have currently is quite nice because it doesn't > require > > >> > >> developers to fiddle around with Javascript as long as they are > not > > >> > >> working on the webfrontend. If they do, they can simply use NPM > and > > >> > >> Bower which install the listed dependencies. The disadvantage is > a > > >> > >> slight increase of our repository because we commit a new > > "vendor.js" > > >> > >> for every recompile, but that is negligible in my opinion. > > >> > >> > > >> > >> It would be cleaner to build the Javascript parts in the Maven > > build > > >> > >> process but it will complicate the build process further. > Frankly, > > I > > >> > >> don't see it'll add much value. Considering the fragility of our > > build > > >> > >> system I might be a bit conservative here :) > > >> > >> > > >> > >> Cheers, > > >> > >> Max > > >> > >> > > >> > >> On Wed, Jun 29, 2016 at 5:17 PM, Till Rohrmann < > > [hidden email]> > > >> > >> wrote: > > >> > >> > Hi Flink community, > > >> > >> > > > >> > >> > while reviewing the LICENSE and NOTICE file of Apache Flink, I > > >> noticed > > >> > >> that > > >> > >> > according to the LICENSE file Flink contains many java script > > files. > > >> > >> > However, tracking the corresponding files back was not so easy, > > >> > because > > >> > >> > they are actually all merged into > > >> > >> > flink-runtime-web/web-dashboard/web/js/vendor.js. Vendor.js is > a > > >> large > > >> > >> java > > >> > >> > script file which is part of the result of the bower build > > process. > > >> > >> > > > >> > >> > I was wondering why we added a "binary" file which is > > auto-generated > > >> > to > > >> > >> our > > >> > >> > code base. Was/Is there any specific reason for it? > > >> > >> > > > >> > >> > The problem is that the java script files contained in the > > vendor.js > > >> > are > > >> > >> > not exactly the set of declared bower dependencies in > > >> > >> > flink-runtime-web/web-dashboard/bower.json. I suspect that also > > >> > >> transitive > > >> > >> > dependencies will be included. At least that is my only > > explanation > > >> > why > > >> > >> > we're referencing lodash.js, for example, in our LICENSE file > > (you > > >> can > > >> > >> find > > >> > >> > it deeply hidden in the vendor.js file). > > >> > >> > > > >> > >> > Wouldn't it be easier to auto-generate the web-dashboard > related > > >> files > > >> > >> > while building Flink? This would not only clean up our > repository > > >> but > > >> > >> also > > >> > >> > make the traceability of contained 3rd party code in our code > > base > > >> > much > > >> > >> > easier. Maybe this maven plugin [1] could help us. > > >> > >> > > > >> > >> > [1] https://github.com/eirslett/frontend-maven-plugin > > >> > >> > > > >> > >> > Cheers, > > >> > >> > Till > > >> > >> > > >> > > > >> > > > |
On Fri, Jul 1, 2016 at 6:50 PM, Aljoscha Krettek <[hidden email]> wrote:
> Hmm, this sounds like we should also have a proper LICENSE/NOTICE for our > binary releases. True... to quote the linked ASF page: "As far as LICENSE and NOTICE are concerned, only bundled bits matter." |
I talked to Stephan and he pointed out that the flink-dist binary file,
which is Flink's fat jar, is not part of the official Flink release. We do offer to download this file as part of a zip file from the Flink website. However, this is only for convenience. In contrast to that, other binary files, such as the jars which are released on maven-central, are part of the official release. Consequently, these files must contain appropriate LICENSE and NOTICE files. If I remember correctly, then this is relevant for modules which shade dependencies away, because the shaded dependency is then contained in the jar. That was also the reason why we removed the license entries for all the dependencies contained in flink-dist. Given that this reasoning is sound, we can keep the LICENSE and NOTICE file as it is modulo the changes we introduced between Flink 1.0 and 1.1. On Sat, Jul 2, 2016 at 6:07 PM, Ufuk Celebi <[hidden email]> wrote: > On Fri, Jul 1, 2016 at 6:50 PM, Aljoscha Krettek <[hidden email]> > wrote: > > Hmm, this sounds like we should also have a proper LICENSE/NOTICE for our > > binary releases. > > True... to quote the linked ASF page: "As far as LICENSE and NOTICE > are concerned, only bundled bits matter." > |
That's good news. :-) Thanks for looking into it Till and Stephan.
On Tue, Jul 5, 2016 at 5:37 PM, Till Rohrmann <[hidden email]> wrote: > I talked to Stephan and he pointed out that the flink-dist binary file, > which is Flink's fat jar, is not part of the official Flink release. We do > offer to download this file as part of a zip file from the Flink website. > However, this is only for convenience. In contrast to that, other binary > files, such as the jars which are released on maven-central, are part of > the official release. Consequently, these files must contain appropriate > LICENSE and NOTICE files. If I remember correctly, then this is relevant > for modules which shade dependencies away, because the shaded dependency is > then contained in the jar. > > That was also the reason why we removed the license entries for all the > dependencies contained in flink-dist. Given that this reasoning is sound, > we can keep the LICENSE and NOTICE file as it is modulo the changes we > introduced between Flink 1.0 and 1.1. > > On Sat, Jul 2, 2016 at 6:07 PM, Ufuk Celebi <[hidden email]> wrote: > >> On Fri, Jul 1, 2016 at 6:50 PM, Aljoscha Krettek <[hidden email]> >> wrote: >> > Hmm, this sounds like we should also have a proper LICENSE/NOTICE for our >> > binary releases. >> >> True... to quote the linked ASF page: "As far as LICENSE and NOTICE >> are concerned, only bundled bits matter." >> |
Free forum by Nabble | Edit this page |