Repository navigation
v1.5.3 - #620
v1.5.3#620
Conversation
…MS. Implement new framework for specifying learning rate as a product of named factors - old configs use hparams.learning_rate_schedule="legacy" and should not be affected (except for transformer_tpu, which uses Adafactor). Move learning rate and Adafactor code to their own files. PiperOrigin-RevId: 186185690
PiperOrigin-RevId: 186256681
PiperOrigin-RevId: 186258236
PiperOrigin-RevId: 186260106
PiperOrigin-RevId: 186321114
PiperOrigin-RevId: 186353744
PiperOrigin-RevId: 186364651
…ments, this PiperOrigin-RevId: 186389799
PiperOrigin-RevId: 186490200
PiperOrigin-RevId: 186513041
PiperOrigin-RevId: 186537741
PiperOrigin-RevId: 186541705
… packages PiperOrigin-RevId: 186645840
…fix bug in static version of ones_matrix_band_part. PiperOrigin-RevId: 186668205
PiperOrigin-RevId: 186681397
PiperOrigin-RevId: 186697250
PiperOrigin-RevId: 186702331
PiperOrigin-RevId: 186704921
…f Transformer. PiperOrigin-RevId: 186915433
PiperOrigin-RevId: 187055251
PiperOrigin-RevId: 187055306
…m machine types when running on CPU PiperOrigin-RevId: 187056322
|
We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for the commit author(s). If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google. |
|
Good to go. |
| flags.DEFINE_string("schedule", "continuous_train_and_eval", | ||
| "Method of Experiment to run.") | ||
| flags.DEFINE_integer("eval_steps", 10000, | ||
| flags.DEFINE_integer("eval_steps", 100, |
There was a problem hiding this comment.
Why? Is there any other usage of the dev set than for development evaluation?
There was a problem hiding this comment.
Try to reduce the number of log statements on error.
There was a problem hiding this comment.
What about silencing the end-of-sequence exception? Or what about using 0="the whole dev set" as the default and measuring its size (number of batches) first?
There was a problem hiding this comment.
Have a way of measuring its size with all the filtering and bucketing and everything that might be going on?
Yes. Simply iterate through all the data before the first evaluation and remember how many steps were needed. I am no tf.Dataset expert, but it feels strange there is no way to iterate through all the date (one epoch) without this hack with an arbitrarily high eval_steps and a bunch of end-of-sequence exceptions.
|
|
||
| @registry.register_hparams | ||
| def afx_base(): | ||
| """Baseline - no momentum, beta=0.999.""" |
|
Have a way of measuring its size with all the filtering and bucketing and
everything that might be going on?
…On Mon, Feb 26, 2018 at 3:36 PM Martin Popel ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In tensor2tensor/bin/t2t_trainer.py
<#620 (comment)>
:
> @@ -65,7 +65,7 @@
flags.DEFINE_string("output_dir", "", "Base output directory for run.")
flags.DEFINE_string("schedule", "continuous_train_and_eval",
"Method of Experiment to run.")
- flags.DEFINE_integer("eval_steps", 10000,
+ flags.DEFINE_integer("eval_steps", 100,
What about silencing the end-of-sequence exception? Or what about using
0="the whole dev set" as the default and measuring its size (number of
batches) first?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#620 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABEGWzZiuwiC2cXiAmEWFhkhdIpLoMhWks5tY0AagaJpZM4ST6ep>
.
|
No description provided.