Skip to content
This repository was archived by the owner on Jul 7, 2023. It is now read-only.

v1.5.3 - #620

Merged
lukaszkaiser merged 22 commits into
tensorflow:masterfrom
rsepassi:push
Feb 26, 2018
Merged

v1.5.3#620
lukaszkaiser merged 22 commits into
tensorflow:masterfrom
rsepassi:push

Conversation

@rsepassi

Copy link
Copy Markdown
Contributor

No description provided.

nshazeer and others added 22 commits February 26, 2018 10:07
…MS. Implement new framework for specifying learning rate as a product of named factors - old configs use hparams.learning_rate_schedule="legacy" and should not be affected (except for transformer_tpu, which uses Adafactor). Move learning rate and Adafactor code to their own files.

PiperOrigin-RevId: 186185690
PiperOrigin-RevId: 186258236
PiperOrigin-RevId: 186321114
PiperOrigin-RevId: 186353744
PiperOrigin-RevId: 186364651
PiperOrigin-RevId: 186513041
PiperOrigin-RevId: 186541705
…fix bug in static version of ones_matrix_band_part.

PiperOrigin-RevId: 186668205
PiperOrigin-RevId: 186681397
PiperOrigin-RevId: 186697250
PiperOrigin-RevId: 186702331
PiperOrigin-RevId: 186704921
PiperOrigin-RevId: 187055306
…m machine types when running on CPU

PiperOrigin-RevId: 187056322
@googlebot

Copy link
Copy Markdown

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for the commit author(s). If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again. If the bot doesn't comment, it means it doesn't think anything has changed.

@rsepassi

Copy link
Copy Markdown
Contributor Author

Good to go.

flags.DEFINE_string("schedule", "continuous_train_and_eval",
"Method of Experiment to run.")
flags.DEFINE_integer("eval_steps", 10000,
flags.DEFINE_integer("eval_steps", 100,

@martinpopel martinpopel Feb 26, 2018 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? Is there any other usage of the dev set than for development evaluation?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to reduce the number of log statements on error.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about silencing the end-of-sequence exception? Or what about using 0="the whole dev set" as the default and measuring its size (number of batches) first?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a way of measuring its size with all the filtering and bucketing and everything that might be going on?

Yes. Simply iterate through all the data before the first evaluation and remember how many steps were needed. I am no tf.Dataset expert, but it feels strange there is no way to iterate through all the date (one epoch) without this hack with an arbitrarily high eval_steps and a bunch of end-of-sequence exceptions.

@lukaszkaiser lukaszkaiser left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

@lukaszkaiser
lukaszkaiser merged commit 5c0d89e into tensorflow:master Feb 26, 2018

@registry.register_hparams
def afx_base():
"""Baseline - no momentum, beta=0.999."""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

beta2=0.999

@rsepassi

rsepassi commented Feb 27, 2018 via email

Copy link
Copy Markdown
Contributor Author

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants