Evolution is a slow algorithm, but it had access to an absurd amount of compute (all neuronal organic matter on Earth) and environment simulation (all of physical reality on Earth) when discovering us; so the discovery of the algorithms/architectures/principles in our heads shouldn't be viewed as trivial.
@Darmani Can you think of any ways traditional program synthesis techniques could be combined with machine learning to perform #2? Assume your system has access to a large amount of practice problems/solutions to train with.
According to this paper https://arxiv.org/pdf/1606.06565.pdf, OpenAI (John Schulman and Paul Christiano) explicitly indicate interest in researching transparency and strongly support work on fairness.
usually zero padding is used; a max_input_length is set somewhere in the code and a number of zeros equal to (max_input_ length - actual_number_of_words_in_input) is appended to the array of input word_ids so that all input sentences have the same length.
The scaling laws have broken: https://arxiv.org/abs/2210.14891