Website and Documentation Infrastructure

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Website and Documentation Infrastructure

Stephan Ewen
I would like to kick off a discussion concerning the project website.

Ufuk, Robert, Fabian, Kostas, and me had a chat today about the website
(and other online contents) and the infrastructure to host them.

As an outcome of that discussion we would propose the following. Please let
us know what you think of it.


1) Project Website with general overview.

Similar to what Mahout offers as a start page. Information about the
project, basic introductions to what flink is, links, status, downloads,
community, infrastructure.

We would like to create the website in using markdown & jekyll and keep
both the markdown files and the rendered html files in the website SVN.

2) "How to contribute" infos

As a part of the website, we would like to include a tutorial on how to
start contributing. This section would include

  - Short summary about how to fork and create pull requests against the
github mirror. Infos how to build and test the code (possibly how to
connect testing infrastructure to your fork)

  - Coding guidelines and how to verify correctness (tests, licences,
headers, checkstyle)

  - A list of issues that would make good "starter" issues in order to
learn the process. Some of us have started using labels in JIRA to mark
such issues, we could simply link a JIRA query there.

Similar to the remainder of the website, this would go into the website SVN
as jekyll flavored markdown and rendered html.


3) Documentation for Flink users.

These docs should explain how to use the system (setup, examples, APIs,
...) We would start with what the current stratosphere website has under
"docs" (http://stratosphere.eu/docs/0.5/) and "quickstart" (
http://stratosphere.eu/quickstart/) and adjust the contents to the current
state and name of the project.

Our thought was to represent all documentation in markdown and make it part
of the code. That way changes to the code and the relevant docs can be done
together. Versioning the code also versions the documentation
automatically. Requests to update the docs come as pull requests or patches
in the same way as fixes for the code come.

We would deploy rendered html files from the documentation once per release
to the website.


4) Documentation about the system internals.

These docs would be relevant to developers, contributors, and others
interested in learning how Flink works. We were thinking about architecture
diagrams for different components and guides like "how to add an operator".

We thought about adding these system internal docs to git repository.
Similarly as the user docs, they could evolve with the code and would be
easy to update or everybody.


5) JavaDocs

The JavaDocs are generated on every maven build anyways, would be pushed
into the SVN for releases.


6) Blog

The blog would be part of the website and come in the same way as markdown.


In addition, we has some thoughts about what to do with snapshot versions
of the documentation and JavaDocs. I leave that to a separate discussion
thread, though.


We are happy to get your comments and what experiences you made with those
approaches.


Stephan
Reply | Threaded
Open this post in threaded view
|

Re: Website and Documentation Infrastructure

Henry Saputra
I am not sure about the blog. I would assume the blog content would
reside somewhere else and the website just refer to them?

On Thu, Jun 19, 2014 at 4:08 PM, Stephan Ewen <[hidden email]> wrote:

> I would like to kick off a discussion concerning the project website.
>
> Ufuk, Robert, Fabian, Kostas, and me had a chat today about the website
> (and other online contents) and the infrastructure to host them.
>
> As an outcome of that discussion we would propose the following. Please let
> us know what you think of it.
>
>
> 1) Project Website with general overview.
>
> Similar to what Mahout offers as a start page. Information about the
> project, basic introductions to what flink is, links, status, downloads,
> community, infrastructure.
>
> We would like to create the website in using markdown & jekyll and keep
> both the markdown files and the rendered html files in the website SVN.
>
> 2) "How to contribute" infos
>
> As a part of the website, we would like to include a tutorial on how to
> start contributing. This section would include
>
>   - Short summary about how to fork and create pull requests against the
> github mirror. Infos how to build and test the code (possibly how to
> connect testing infrastructure to your fork)
>
>   - Coding guidelines and how to verify correctness (tests, licences,
> headers, checkstyle)
>
>   - A list of issues that would make good "starter" issues in order to
> learn the process. Some of us have started using labels in JIRA to mark
> such issues, we could simply link a JIRA query there.
>
> Similar to the remainder of the website, this would go into the website SVN
> as jekyll flavored markdown and rendered html.
>
>
> 3) Documentation for Flink users.
>
> These docs should explain how to use the system (setup, examples, APIs,
> ...) We would start with what the current stratosphere website has under
> "docs" (http://stratosphere.eu/docs/0.5/) and "quickstart" (
> http://stratosphere.eu/quickstart/) and adjust the contents to the current
> state and name of the project.
>
> Our thought was to represent all documentation in markdown and make it part
> of the code. That way changes to the code and the relevant docs can be done
> together. Versioning the code also versions the documentation
> automatically. Requests to update the docs come as pull requests or patches
> in the same way as fixes for the code come.
>
> We would deploy rendered html files from the documentation once per release
> to the website.
>
>
> 4) Documentation about the system internals.
>
> These docs would be relevant to developers, contributors, and others
> interested in learning how Flink works. We were thinking about architecture
> diagrams for different components and guides like "how to add an operator".
>
> We thought about adding these system internal docs to git repository.
> Similarly as the user docs, they could evolve with the code and would be
> easy to update or everybody.
>
>
> 5) JavaDocs
>
> The JavaDocs are generated on every maven build anyways, would be pushed
> into the SVN for releases.
>
>
> 6) Blog
>
> The blog would be part of the website and come in the same way as markdown.
>
>
> In addition, we has some thoughts about what to do with snapshot versions
> of the documentation and JavaDocs. I leave that to a separate discussion
> thread, though.
>
>
> We are happy to get your comments and what experiences you made with those
> approaches.
>
>
> Stephan
Reply | Threaded
Open this post in threaded view
|

Re: Website and Documentation Infrastructure

Ufuk Celebi
Hey Henry,

On 22 Jun 2014, at 07:54, Henry Saputra <[hidden email]> wrote:

> I am not sure about the blog. I would assume the blog content would
> reside somewhere else and the website just refer to them?

we had good experiences with Jekyll in the past. Blog entries are basically a folder of Markdown files in SVN. Jekyll generates the static HTML files to the SVN contents directory to publish stuff.

I would really like it if we could keep it that way in order to have everything website related in a single place. Your proposal would be to use https://blogs.apache.org, right?

-- Ufuk
Reply | Threaded
Open this post in threaded view
|

Re: Website and Documentation Infrastructure

Henry Saputra
Hi Ufuk,

Thanks for the response. It is also how what kind of features the blog
page should have, like RSS feed, blog summaries, etc.

My thinking was to link the blogs to the Apache Flink wiki (so each
blog entry corresponds to a wiki page). This way, non-committers could
still create blog pages, and also the website could add link to
external blogs from other websites.

Just my 2-cents.

- Henry



On Mon, Jun 23, 2014 at 12:06 AM, Ufuk Celebi <[hidden email]> wrote:

> Hey Henry,
>
> On 22 Jun 2014, at 07:54, Henry Saputra <[hidden email]> wrote:
>
>> I am not sure about the blog. I would assume the blog content would
>> reside somewhere else and the website just refer to them?
>
> we had good experiences with Jekyll in the past. Blog entries are basically a folder of Markdown files in SVN. Jekyll generates the static HTML files to the SVN contents directory to publish stuff.
>
> I would really like it if we could keep it that way in order to have everything website related in a single place. Your proposal would be to use https://blogs.apache.org, right?
>
> -- Ufuk
Reply | Threaded
Open this post in threaded view
|

Re: Website and Documentation Infrastructure

Robert Metzger
I think RSS feeds are supported by Jekyll (see:
http://stratosphere.eu/feed.xml). There are some plugins to generate other
stuff known from blogs (categories, tag-clouds etc.)



On Mon, Jun 23, 2014 at 9:52 AM, Henry Saputra <[hidden email]>
wrote:

> Hi Ufuk,
>
> Thanks for the response. It is also how what kind of features the blog
> page should have, like RSS feed, blog summaries, etc.
>
> My thinking was to link the blogs to the Apache Flink wiki (so each
> blog entry corresponds to a wiki page). This way, non-committers could
> still create blog pages, and also the website could add link to
> external blogs from other websites.
>
> Just my 2-cents.
>
> - Henry
>
>
>
> On Mon, Jun 23, 2014 at 12:06 AM, Ufuk Celebi <[hidden email]>
> wrote:
> > Hey Henry,
> >
> > On 22 Jun 2014, at 07:54, Henry Saputra <[hidden email]> wrote:
> >
> >> I am not sure about the blog. I would assume the blog content would
> >> reside somewhere else and the website just refer to them?
> >
> > we had good experiences with Jekyll in the past. Blog entries are
> basically a folder of Markdown files in SVN. Jekyll generates the static
> HTML files to the SVN contents directory to publish stuff.
> >
> > I would really like it if we could keep it that way in order to have
> everything website related in a single place. Your proposal would be to use
> https://blogs.apache.org, right?
> >
> > -- Ufuk
>
Reply | Threaded
Open this post in threaded view
|

Re: Website and Documentation Infrastructure

Stephan Ewen
Henry's point is also to have an easy way of creating / editing posts, for
non committers. That is a good point, indeed.

I guess markdown/jekyll/html looks prettier, but the wiki is more open to
others. Can we combine the two in some way?
Reply | Threaded
Open this post in threaded view
|

Re: Website and Documentation Infrastructure

Robert Metzger
I would suggest that people use the Wiki to draft a blog post and a
committer than adds the final text to the blog on the website (its just
adding one markdown file)


On Mon, Jun 23, 2014 at 3:14 PM, Stephan Ewen <[hidden email]> wrote:

> Henry's point is also to have an easy way of creating / editing posts, for
> non committers. That is a good point, indeed.
>
> I guess markdown/jekyll/html looks prettier, but the wiki is more open to
> others. Can we combine the two in some way?
>
Reply | Threaded
Open this post in threaded view
|

Re: Website and Documentation Infrastructure

Henry Saputra
Another way is to use the Blogs page to just contains links to the real blogs.

So, all Flink contributors would add blog in Flink wiki or even
external site and the project website's Blog page is just list all the
links to them.

Thoughts?

- Henry

On Wed, Jul 2, 2014 at 7:33 AM, Robert Metzger <[hidden email]> wrote:

> I would suggest that people use the Wiki to draft a blog post and a
> committer than adds the final text to the blog on the website (its just
> adding one markdown file)
>
>
> On Mon, Jun 23, 2014 at 3:14 PM, Stephan Ewen <[hidden email]> wrote:
>
>> Henry's point is also to have an easy way of creating / editing posts, for
>> non committers. That is a good point, indeed.
>>
>> I guess markdown/jekyll/html looks prettier, but the wiki is more open to
>> others. Can we combine the two in some way?
>>