Hello
Some operators call system.currentTimeMillis() frequently and it is cost performance. I want to use a thread to call System.currentTimeMillis and update a long variable millTime. All other module do not need call System.currentTimeMillis() and can get millTime directly. I want to know your suggestion. Thank you very much. Regards Fengbin Fang |
Hey Fengbin,
did you run a program and noticed some kind of performance degradation because of this? If yes, could you provide some details? If not, I would suggest to not do this. I can see how this "improves" performance in micro benchmarks, but not how it would affect the overall performance of the whole system with real programs running. – Ufuk On Wed, Aug 12, 2015 at 11:33 AM, Fangfengbin <[hidden email]> wrote: > Hello > > > > Some operators call system.currentTimeMillis() frequently and it is cost > performance. > > I want to use a thread to call System.currentTimeMillis and update a long > variable millTime. All other module do not need call > System.currentTimeMillis() and can get millTime directly. > > I want to know your suggestion. Thank you very much. > > > > > > Regards > > Fengbin Fang > > > |
In reply to this post by Fangfengbin
-.-
if you look at into this issue that you opened https://issues.apache.org/jira/browse/FLINK-2471 you were given opinions from 2 separate people, with arguments, that the performance improvement(TBD) is either a) nonexistant or b) negligible. All you did was disregard those basically saying "no I'm right.", and not much else. And now we're here starting at square one. SO, my suggestion is this: You said you could test the performance, so go ahead. Then we can talk. Regards, a slightly disgruntled Chesnay Schepler On 12.08.2015 11:33, Fangfengbin wrote: > Hello > > > > Some operators call system.currentTimeMillis() frequently and it is cost performance. > > I want to use a thread to call System.currentTimeMillis and update a long variable millTime. All other module do not need call System.currentTimeMillis() and can get millTime directly. > > I want to know your suggestion. Thank you very much. > > > > > > Regards > > Fengbin Fang > > > |
I second Ufuk and Chensnay. Please provide us with a benchmark. I have a
hard time to believe your implementation, along with the overhead that comes with it, will improve the streaming performance. Please, feel free to prove us wrong :) On Wed, Aug 12, 2015 at 11:48 AM, Chesnay Schepler < [hidden email]> wrote: > -.- > > if you look at into this issue that you opened > https://issues.apache.org/jira/browse/FLINK-2471 you were given opinions > from 2 separate people, with arguments, that the performance > improvement(TBD) is either a) nonexistant or b) negligible. All you did was > disregard those basically saying "no I'm right.", and not much else. And > now we're here starting at square one. > > SO, my suggestion is this: You said you could test the performance, so go > ahead. Then we can talk. > > Regards, > a slightly disgruntled Chesnay Schepler > > > On 12.08.2015 11:33, Fangfengbin wrote: > >> Hello >> >> >> >> Some operators call system.currentTimeMillis() frequently and it is cost >> performance. >> >> I want to use a thread to call System.currentTimeMillis and update a long >> variable millTime. All other module do not need call >> System.currentTimeMillis() and can get millTime directly. >> >> I want to know your suggestion. Thank you very much. >> >> >> >> >> >> Regards >> >> Fengbin Fang >> >> >> >> > |
@ffbin: Would this be a JVM singleton that updates a static field every
millisecond? It is hard to say at this point whether this eliminates a bottleneck, but I think it is fine to try. To evaluate this, you could use a streaming job that attaches a timestamp to every record, and measure the different throughput with and without this service. I curious what the results are! On Wed, Aug 12, 2015 at 2:39 PM, Maximilian Michels <[hidden email]> wrote: > I second Ufuk and Chensnay. Please provide us with a benchmark. I have a > hard time to believe your implementation, along with the overhead that > comes with it, will improve the streaming performance. > > Please, feel free to prove us wrong :) > > On Wed, Aug 12, 2015 at 11:48 AM, Chesnay Schepler < > [hidden email]> wrote: > > > -.- > > > > if you look at into this issue that you opened > > https://issues.apache.org/jira/browse/FLINK-2471 you were given opinions > > from 2 separate people, with arguments, that the performance > > improvement(TBD) is either a) nonexistant or b) negligible. All you did > was > > disregard those basically saying "no I'm right.", and not much else. And > > now we're here starting at square one. > > > > SO, my suggestion is this: You said you could test the performance, so go > > ahead. Then we can talk. > > > > Regards, > > a slightly disgruntled Chesnay Schepler > > > > > > On 12.08.2015 11:33, Fangfengbin wrote: > > > >> Hello > >> > >> > >> > >> Some operators call system.currentTimeMillis() frequently and it is cost > >> performance. > >> > >> I want to use a thread to call System.currentTimeMillis and update a > long > >> variable millTime. All other module do not need call > >> System.currentTimeMillis() and can get millTime directly. > >> > >> I want to know your suggestion. Thank you very much. > >> > >> > >> > >> > >> > >> Regards > >> > >> Fengbin Fang > >> > >> > >> > >> > > > |
On Wed, Aug 12, 2015 at 3:57 PM, Stephan Ewen <[hidden email]> wrote:
> I curious what the results are! Same here! :-) |
In reply to this post by Stephan Ewen
Hello!
I have a test about cost of System.currentTimeMillis() and my CLOCK.currentTimeMillis() .( My clock will be a JVM singleton) Call currentTimeMillis function 100000000 times, System.currentTimeMillis() need about 1902ms and my CLOCK.currentTimeMillis() only need 119ms. The function performance is up to about 15 times. Next I will use a streaming job to test it. This is my code: ######### class MillisecondClock { private long rate = 0; private volatile long now = 0; private volatile boolean isRunning = true; private MillisecondClock(long rate) { this.rate = rate; this.now = System.currentTimeMillis(); start(); } private void start() { new Thread(new Runnable() { @Override public void run() { while(isRunning) { try { Thread.sleep(rate); } catch (InterruptedException e) { e.printStackTrace(); } now = System.currentTimeMillis(); } } }).start(); } public long currentTimeMillis() { return now; } public static final MillisecondClock CLOCK = new MillisecondClock(1); public static void main(String[] args) { long nowTime = 0; //test system currentTimeMillis long startTime = System.currentTimeMillis(); long endTime; for(int i = 0;i < 100000000;i++) { nowTime = System.currentTimeMillis(); } endTime = System.currentTimeMillis(); System.out.println("Time cost: " + (endTime-startTime)); //test my currentTimeMillis CLOCK.start(); startTime = System.currentTimeMillis(); for(int i = 0;i < 100000000;i++) { nowTime = CLOCK.currentTimeMillis(); } endTime = System.currentTimeMillis(); System.out.println("Time cost: " + (endTime-startTime)); } } ######## Regards Fengbin Fang |
Thank you Fengbin Fang for doing this microbenchmark!
The numbers clearly show that your approach is a lot faster. I'm curious if this does also affect the performance of a complete data flow. Looking forward to your results, Fabian 2015-08-13 8:35 GMT+02:00 Fangfengbin <[hidden email]>: > Hello! > > I have a test about cost of System.currentTimeMillis() and my > CLOCK.currentTimeMillis() .( My clock will be a JVM singleton) > Call currentTimeMillis function 100000000 times, > System.currentTimeMillis() need about 1902ms and my > CLOCK.currentTimeMillis() only need 119ms. > The function performance is up to about 15 times. > Next I will use a streaming job to test it. > > This is my code: > ######### > class MillisecondClock { > private long rate = 0; > private volatile long now = 0; > > private volatile boolean isRunning = true; > > private MillisecondClock(long rate) { > this.rate = rate; > this.now = System.currentTimeMillis(); > start(); > } > > private void start() { > new Thread(new Runnable() { > @Override > public void run() { > while(isRunning) > { > try { > Thread.sleep(rate); > } catch (InterruptedException e) { > e.printStackTrace(); > } > now = System.currentTimeMillis(); > } > > } > }).start(); > } > > public long currentTimeMillis() { > return now; > } > > public static final MillisecondClock CLOCK = new MillisecondClock(1); > > > public static void main(String[] args) > { > long nowTime = 0; > > //test system currentTimeMillis > long startTime = System.currentTimeMillis(); > long endTime; > for(int i = 0;i < 100000000;i++) > { > nowTime = System.currentTimeMillis(); > } > endTime = System.currentTimeMillis(); > System.out.println("Time cost: " + (endTime-startTime)); > > //test my currentTimeMillis > CLOCK.start(); > startTime = System.currentTimeMillis(); > for(int i = 0;i < 100000000;i++) > { > nowTime = CLOCK.currentTimeMillis(); > } > endTime = System.currentTimeMillis(); > System.out.println("Time cost: " + (endTime-startTime)); > > } > } > ######## > > Regards > Fengbin Fang > |
Hi!
Nice idea. It would be good to have this implemented as a kind of service that components can "acquire". A component that needs time can do something like "Clock clock = ClockService.aquire()", which increments a reference count in the central clock service and starts the thread if it has not yet been started. When it does not need the clock any more, it can call "clock.release()". If the reference count reaches 0, the timer thread is stopped. Additionally, we could use the "finalize()" method as a safety net to release a clock in case the user forgot to release it. Greetings, Stephan On Fri, Aug 14, 2015 at 10:56 AM, Fabian Hueske <[hidden email]> wrote: > Thank you Fengbin Fang for doing this microbenchmark! > The numbers clearly show that your approach is a lot faster. > > I'm curious if this does also affect the performance of a complete data > flow. > > Looking forward to your results, > Fabian > > 2015-08-13 8:35 GMT+02:00 Fangfengbin <[hidden email]>: > > > Hello! > > > > I have a test about cost of System.currentTimeMillis() and my > > CLOCK.currentTimeMillis() .( My clock will be a JVM singleton) > > Call currentTimeMillis function 100000000 times, > > System.currentTimeMillis() need about 1902ms and my > > CLOCK.currentTimeMillis() only need 119ms. > > The function performance is up to about 15 times. > > Next I will use a streaming job to test it. > > > > This is my code: > > ######### > > class MillisecondClock { > > private long rate = 0; > > private volatile long now = 0; > > > > private volatile boolean isRunning = true; > > > > private MillisecondClock(long rate) { > > this.rate = rate; > > this.now = System.currentTimeMillis(); > > start(); > > } > > > > private void start() { > > new Thread(new Runnable() { > > @Override > > public void run() { > > while(isRunning) > > { > > try { > > Thread.sleep(rate); > > } catch (InterruptedException e) { > > e.printStackTrace(); > > } > > now = System.currentTimeMillis(); > > } > > > > } > > }).start(); > > } > > > > public long currentTimeMillis() { > > return now; > > } > > > > public static final MillisecondClock CLOCK = new MillisecondClock(1); > > > > > > public static void main(String[] args) > > { > > long nowTime = 0; > > > > //test system currentTimeMillis > > long startTime = System.currentTimeMillis(); > > long endTime; > > for(int i = 0;i < 100000000;i++) > > { > > nowTime = System.currentTimeMillis(); > > } > > endTime = System.currentTimeMillis(); > > System.out.println("Time cost: " + (endTime-startTime)); > > > > //test my currentTimeMillis > > CLOCK.start(); > > startTime = System.currentTimeMillis(); > > for(int i = 0;i < 100000000;i++) > > { > > nowTime = CLOCK.currentTimeMillis(); > > } > > endTime = System.currentTimeMillis(); > > System.out.println("Time cost: " + (endTime-startTime)); > > > > } > > } > > ######## > > > > Regards > > Fengbin Fang > > > |
A big part of the cost of currentTimeMillis() is that it does a lot of work
to make sure that the time is really time. If you only need a monotonic timer, nanoTime() might be what you want instead of currentTimeMillis(). nanoTime() is particularly handy when you want to avoid issues to do with leap seconds and it is particularly bad when you really want the current time. On Fri, Aug 14, 2015 at 2:03 AM, Stephan Ewen <[hidden email]> wrote: > Hi! > > Nice idea. > > It would be good to have this implemented as a kind of service that > components can "acquire". > > A component that needs time can do something like "Clock clock = > ClockService.aquire()", which increments a reference count in the central > clock service and starts the thread if it has not yet been started. > When it does not need the clock any more, it can call "clock.release()". If > the reference count reaches 0, the timer thread is stopped. > > Additionally, we could use the "finalize()" method as a safety net to > release a clock in case the user forgot to release it. > > Greetings, > Stephan > > > On Fri, Aug 14, 2015 at 10:56 AM, Fabian Hueske <[hidden email]> wrote: > > > Thank you Fengbin Fang for doing this microbenchmark! > > The numbers clearly show that your approach is a lot faster. > > > > I'm curious if this does also affect the performance of a complete data > > flow. > > > > Looking forward to your results, > > Fabian > > > > 2015-08-13 8:35 GMT+02:00 Fangfengbin <[hidden email]>: > > > > > Hello! > > > > > > I have a test about cost of System.currentTimeMillis() and my > > > CLOCK.currentTimeMillis() .( My clock will be a JVM singleton) > > > Call currentTimeMillis function 100000000 times, > > > System.currentTimeMillis() need about 1902ms and my > > > CLOCK.currentTimeMillis() only need 119ms. > > > The function performance is up to about 15 times. > > > Next I will use a streaming job to test it. > > > > > > This is my code: > > > ######### > > > class MillisecondClock { > > > private long rate = 0; > > > private volatile long now = 0; > > > > > > private volatile boolean isRunning = true; > > > > > > private MillisecondClock(long rate) { > > > this.rate = rate; > > > this.now = System.currentTimeMillis(); > > > start(); > > > } > > > > > > private void start() { > > > new Thread(new Runnable() { > > > @Override > > > public void run() { > > > while(isRunning) > > > { > > > try { > > > Thread.sleep(rate); > > > } catch (InterruptedException e) { > > > e.printStackTrace(); > > > } > > > now = System.currentTimeMillis(); > > > } > > > > > > } > > > }).start(); > > > } > > > > > > public long currentTimeMillis() { > > > return now; > > > } > > > > > > public static final MillisecondClock CLOCK = new > MillisecondClock(1); > > > > > > > > > public static void main(String[] args) > > > { > > > long nowTime = 0; > > > > > > //test system currentTimeMillis > > > long startTime = System.currentTimeMillis(); > > > long endTime; > > > for(int i = 0;i < 100000000;i++) > > > { > > > nowTime = System.currentTimeMillis(); > > > } > > > endTime = System.currentTimeMillis(); > > > System.out.println("Time cost: " + (endTime-startTime)); > > > > > > //test my currentTimeMillis > > > CLOCK.start(); > > > startTime = System.currentTimeMillis(); > > > for(int i = 0;i < 100000000;i++) > > > { > > > nowTime = CLOCK.currentTimeMillis(); > > > } > > > endTime = System.currentTimeMillis(); > > > System.out.println("Time cost: " + (endTime-startTime)); > > > > > > } > > > } > > > ######## > > > > > > Regards > > > Fengbin Fang > > > > > > |
In many cases, we actually need a proper timestamp (for example for
timestamps in streaming records). I just ran a small micro-benchmark (Java 8, Linux kernel 3.11, 64bit VM) Making 5,000,000 function calls System.currentTimeMillis() took: 13240msecs System.nanoTime() took: 13355msecs At least on my setup, they seem similar. Not sure about other setups, though. I am running in a Linux VM, anything that accesses hardware (like the clock) may be different on bare metal. On Fri, Aug 14, 2015 at 5:40 PM, Ted Dunning <[hidden email]> wrote: > A big part of the cost of currentTimeMillis() is that it does a lot of work > to make sure that the time is really time. > > If you only need a monotonic timer, nanoTime() might be what you want > instead of currentTimeMillis(). nanoTime() is particularly handy when you > want to avoid issues to do with leap seconds and it is particularly bad > when you really want the current time. > > > On Fri, Aug 14, 2015 at 2:03 AM, Stephan Ewen <[hidden email]> wrote: > > > Hi! > > > > Nice idea. > > > > It would be good to have this implemented as a kind of service that > > components can "acquire". > > > > A component that needs time can do something like "Clock clock = > > ClockService.aquire()", which increments a reference count in the central > > clock service and starts the thread if it has not yet been started. > > When it does not need the clock any more, it can call "clock.release()". > If > > the reference count reaches 0, the timer thread is stopped. > > > > Additionally, we could use the "finalize()" method as a safety net to > > release a clock in case the user forgot to release it. > > > > Greetings, > > Stephan > > > > > > On Fri, Aug 14, 2015 at 10:56 AM, Fabian Hueske <[hidden email]> > wrote: > > > > > Thank you Fengbin Fang for doing this microbenchmark! > > > The numbers clearly show that your approach is a lot faster. > > > > > > I'm curious if this does also affect the performance of a complete data > > > flow. > > > > > > Looking forward to your results, > > > Fabian > > > > > > 2015-08-13 8:35 GMT+02:00 Fangfengbin <[hidden email]>: > > > > > > > Hello! > > > > > > > > I have a test about cost of System.currentTimeMillis() and my > > > > CLOCK.currentTimeMillis() .( My clock will be a JVM singleton) > > > > Call currentTimeMillis function 100000000 times, > > > > System.currentTimeMillis() need about 1902ms and my > > > > CLOCK.currentTimeMillis() only need 119ms. > > > > The function performance is up to about 15 times. > > > > Next I will use a streaming job to test it. > > > > > > > > This is my code: > > > > ######### > > > > class MillisecondClock { > > > > private long rate = 0; > > > > private volatile long now = 0; > > > > > > > > private volatile boolean isRunning = true; > > > > > > > > private MillisecondClock(long rate) { > > > > this.rate = rate; > > > > this.now = System.currentTimeMillis(); > > > > start(); > > > > } > > > > > > > > private void start() { > > > > new Thread(new Runnable() { > > > > @Override > > > > public void run() { > > > > while(isRunning) > > > > { > > > > try { > > > > Thread.sleep(rate); > > > > } catch (InterruptedException e) { > > > > e.printStackTrace(); > > > > } > > > > now = System.currentTimeMillis(); > > > > } > > > > > > > > } > > > > }).start(); > > > > } > > > > > > > > public long currentTimeMillis() { > > > > return now; > > > > } > > > > > > > > public static final MillisecondClock CLOCK = new > > MillisecondClock(1); > > > > > > > > > > > > public static void main(String[] args) > > > > { > > > > long nowTime = 0; > > > > > > > > //test system currentTimeMillis > > > > long startTime = System.currentTimeMillis(); > > > > long endTime; > > > > for(int i = 0;i < 100000000;i++) > > > > { > > > > nowTime = System.currentTimeMillis(); > > > > } > > > > endTime = System.currentTimeMillis(); > > > > System.out.println("Time cost: " + (endTime-startTime)); > > > > > > > > //test my currentTimeMillis > > > > CLOCK.start(); > > > > startTime = System.currentTimeMillis(); > > > > for(int i = 0;i < 100000000;i++) > > > > { > > > > nowTime = CLOCK.currentTimeMillis(); > > > > } > > > > endTime = System.currentTimeMillis(); > > > > System.out.println("Time cost: " + (endTime-startTime)); > > > > > > > > } > > > > } > > > > ######## > > > > > > > > Regards > > > > Fengbin Fang > > > > > > > > > > |
Free forum by Nabble | Edit this page |