Skip to content
This repository was archived by the owner on Oct 11, 2023. It is now read-only.
This repository was archived by the owner on Oct 11, 2023. It is now read-only.

Learning rate for NN training  #26

@pamio

Description

@pamio

in tensorflow or torch, I have observed that the learning rate is usually one number like 0.001 or 0.01 etc. But in this tutorial I find that the learning rate for 16 classes is something like this

cntk_lr_per_image = [0.01] * 10 + [0.001] * 5 + [0.0001] which is array of 16 items which is equal to the number of classes. So if I have 80 classes to train should this array cntk_lr_per_image contain 80 learning rate ?

Something like this cntk_lr_per_image = [0.01] * 40 + [0.001] * 30 + [0.0001]*10 ?
I understand what learning rate is and how it impacts the overall learning of the model and there's no straight forward way to choose a learning rate but this tutorial uses an array of learning rates which is what confuses me.

Can you help me with understanding how learning rate us used in this example ?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions