[INFO] updating cached repository mklf/bert_tokenizer
[INFO] running `"git" "fetch" "--all"`
[INFO] [stdout] Fetching origin
[INFO] [stderr] From git://github.com/mklf/bert_tokenizer
[INFO] [stderr]  * branch            HEAD       -> FETCH_HEAD
[INFO] running `"git" "clone" "work/cache/sources/gh/mklf/bert_tokenizer" "work/ex/clippy-test-run/sources/stable/gh/mklf/bert_tokenizer"`
[INFO] [stderr] Cloning into 'work/ex/clippy-test-run/sources/stable/gh/mklf/bert_tokenizer'...
[INFO] [stderr] done.
[INFO] running `"git" "clone" "work/cache/sources/gh/mklf/bert_tokenizer" "work/ex/clippy-test-run/sources/stable+rustflags=-Dclippy%3A%3Ainto_iter_on_array/gh/mklf/bert_tokenizer"`
[INFO] [stderr] Cloning into 'work/ex/clippy-test-run/sources/stable+rustflags=-Dclippy%3A%3Ainto_iter_on_array/gh/mklf/bert_tokenizer'...
[INFO] [stderr] done.
[INFO] running `"git" "rev-parse" "HEAD"`
[INFO] [stdout] db07849c04e6fdf60405fcb0ee97a69acc0a8b85
[INFO] sha for GitHub repo mklf/bert_tokenizer: db07849c04e6fdf60405fcb0ee97a69acc0a8b85
[INFO] validating manifest of mklf/bert_tokenizer on toolchain stable
[INFO] running `"/mnt/big/crater/work/local/cargo-home/bin/cargo" "+stable" "read-manifest" "--manifest-path" "Cargo.toml"`
[INFO] validating manifest of mklf/bert_tokenizer on toolchain stable+rustflags=-Dclippy::into_iter_on_array
[INFO] running `"/mnt/big/crater/work/local/cargo-home/bin/cargo" "+stable" "read-manifest" "--manifest-path" "Cargo.toml"`
[INFO] started frobbing mklf/bert_tokenizer
[INFO] finished frobbing mklf/bert_tokenizer
[INFO] frobbed toml for mklf/bert_tokenizer written to work/ex/clippy-test-run/sources/stable/gh/mklf/bert_tokenizer/Cargo.toml
[INFO] started frobbing mklf/bert_tokenizer
[INFO] finished frobbing mklf/bert_tokenizer
[INFO] frobbed toml for mklf/bert_tokenizer written to work/ex/clippy-test-run/sources/stable+rustflags=-Dclippy%3A%3Ainto_iter_on_array/gh/mklf/bert_tokenizer/Cargo.toml
[INFO] crate mklf/bert_tokenizer has a lockfile. skipping
[INFO] running `"/mnt/big/crater/work/local/cargo-home/bin/cargo" "+stable" "fetch" "--locked" "--manifest-path" "Cargo.toml"`
[INFO] running `"/mnt/big/crater/work/local/cargo-home/bin/cargo" "+stable" "fetch" "--locked" "--manifest-path" "Cargo.toml"`
[INFO] linting mklf/bert_tokenizer against stable for clippy-test-run
[INFO] running `"docker" "create" "-v" "/mnt/big/crater/work/local/target-dirs/clippy-test-run/worker-5/stable:/opt/crater/target:rw,Z" "-v" "/mnt/big/crater/work/ex/clippy-test-run/sources/stable/gh/mklf/bert_tokenizer:/opt/crater/workdir:ro,Z" "-v" "/mnt/big/crater/work/local/cargo-home:/opt/crater/cargo-home:ro,Z" "-v" "/mnt/big/crater/work/local/rustup-home:/opt/crater/rustup-home:ro,Z" "-e" "USER_ID=1000" "-e" "SOURCE_DIR=/opt/crater/workdir" "-e" "MAP_USER_ID=1000" "-e" "CARGO_TARGET_DIR=/opt/crater/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/crater/cargo-home" "-e" "RUSTUP_HOME=/opt/crater/rustup-home" "-w" "/opt/crater/workdir" "-m" "1536M" "--network" "none" "rustops/crates-build-env" "/opt/crater/cargo-home/bin/cargo" "+stable" "clippy" "--frozen" "--all" "--all-targets"`
[INFO] [stdout] 84d9769048f2598c35a317f223e237ce095d013f6cfb2280bfeb082a079e87ba
[INFO] running `"docker" "start" "-a" "84d9769048f2598c35a317f223e237ce095d013f6cfb2280bfeb082a079e87ba"`
[INFO] [stderr]     Checking bert_tokenizer v0.1.0 (/opt/crater/workdir)
[INFO] [stderr] warning: unneeded return statement
[INFO] [stderr]   --> src/tokenization.rs:33:5
[INFO] [stderr]    |
[INFO] [stderr] 33 |     return character.is_control()
[INFO] [stderr]    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: remove `return` as shown: `character.is_control()`
[INFO] [stderr]    |
[INFO] [stderr]    = note: #[warn(clippy::needless_return)] on by default
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
[INFO] [stderr] 
[INFO] [stderr] warning: unneeded return statement
[INFO] [stderr]   --> src/tokenization.rs:58:9
[INFO] [stderr]    |
[INFO] [stderr] 58 | /         return (cp >= 0x4E00 && cp <= 0x9FFF) ||
[INFO] [stderr] 59 | |             (cp >= 0x3400 && cp <= 0x4DBF) ||
[INFO] [stderr] 60 | |             (cp >= 0x20000 && cp <= 0x2A6DF) ||
[INFO] [stderr] 61 | |             (cp >= 0x2A700 && cp <= 0x2B73F) ||
[INFO] [stderr] ...  |
[INFO] [stderr] 64 | |             (cp >= 0xF900 && cp <= 0xFAFF) ||
[INFO] [stderr] 65 | |             (cp >= 0x2F800 && cp <= 0x2FA1F);
[INFO] [stderr]    | |_____________________________________________^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
[INFO] [stderr] help: remove `return` as shown
[INFO] [stderr]    |
[INFO] [stderr] 58 |         (cp >= 0x4E00 && cp <= 0x9FFF) ||
[INFO] [stderr] 59 |             (cp >= 0x3400 && cp <= 0x4DBF) ||
[INFO] [stderr] 60 |             (cp >= 0x20000 && cp <= 0x2A6DF) ||
[INFO] [stderr] 61 |             (cp >= 0x2A700 && cp <= 0x2B73F) ||
[INFO] [stderr] 62 |             (cp >= 0x2B740 && cp <= 0x2B81F) ||
[INFO] [stderr] 63 |             (cp >= 0x2B820 && cp <= 0x2CEAF) ||
[INFO] [stderr]  ...
[INFO] [stderr] 
[INFO] [stderr] warning: unneeded return statement
[INFO] [stderr]    --> src/tokenization.rs:198:9
[INFO] [stderr]     |
[INFO] [stderr] 198 |         return output_tokens;
[INFO] [stderr]     |         ^^^^^^^^^^^^^^^^^^^^^ help: remove `return` as shown: `output_tokens`
[INFO] [stderr]     |
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
[INFO] [stderr] 
[INFO] [stderr] warning: unneeded return statement
[INFO] [stderr]    --> src/tokenization.rs:241:9
[INFO] [stderr]     |
[INFO] [stderr] 241 |         return output_tokens;
[INFO] [stderr]     |         ^^^^^^^^^^^^^^^^^^^^^ help: remove `return` as shown: `output_tokens`
[INFO] [stderr]     |
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
[INFO] [stderr] 
[INFO] [stderr] warning: returning the result of a let binding from a block. Consider returning the expression directly.
[INFO] [stderr]   --> src/ffi.rs:18:5
[INFO] [stderr]    |
[INFO] [stderr] 18 |     error_ptr
[INFO] [stderr]    |     ^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = note: #[warn(clippy::let_and_return)] on by default
[INFO] [stderr] note: this expression can be directly returned
[INFO] [stderr]   --> src/ffi.rs:15:21
[INFO] [stderr]    |
[INFO] [stderr] 15 |       let error_ptr = ERROR_MSG.with(|error_msg|{
[INFO] [stderr]    |  _____________________^
[INFO] [stderr] 16 | |         return error_msg.borrow().as_ptr();
[INFO] [stderr] 17 | |     });
[INFO] [stderr]    | |______^
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#let_and_return
[INFO] [stderr] 
[INFO] [stderr] warning: unneeded return statement
[INFO] [stderr]   --> src/ffi.rs:16:9
[INFO] [stderr]    |
[INFO] [stderr] 16 |         return error_msg.borrow().as_ptr();
[INFO] [stderr]    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: remove `return` as shown: `error_msg.borrow().as_ptr()`
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
[INFO] [stderr] 
[INFO] [stderr] warning: unneeded return statement
[INFO] [stderr]   --> src/tokenization.rs:33:5
[INFO] [stderr]    |
[INFO] [stderr] 33 |     return character.is_control()
[INFO] [stderr]    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: remove `return` as shown: `character.is_control()`
[INFO] [stderr]    |
[INFO] [stderr]    = note: #[warn(clippy::needless_return)] on by default
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
[INFO] [stderr] 
[INFO] [stderr] warning: unneeded return statement
[INFO] [stderr]   --> src/tokenization.rs:58:9
[INFO] [stderr]    |
[INFO] [stderr] 58 | /         return (cp >= 0x4E00 && cp <= 0x9FFF) ||
[INFO] [stderr] 59 | |             (cp >= 0x3400 && cp <= 0x4DBF) ||
[INFO] [stderr] 60 | |             (cp >= 0x20000 && cp <= 0x2A6DF) ||
[INFO] [stderr] 61 | |             (cp >= 0x2A700 && cp <= 0x2B73F) ||
[INFO] [stderr] ...  |
[INFO] [stderr] 64 | |             (cp >= 0xF900 && cp <= 0xFAFF) ||
[INFO] [stderr] 65 | |             (cp >= 0x2F800 && cp <= 0x2FA1F);
[INFO] [stderr]    | |_____________________________________________^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
[INFO] [stderr] help: remove `return` as shown
[INFO] [stderr]    |
[INFO] [stderr] 58 |         (cp >= 0x4E00 && cp <= 0x9FFF) ||
[INFO] [stderr] 59 |             (cp >= 0x3400 && cp <= 0x4DBF) ||
[INFO] [stderr] 60 |             (cp >= 0x20000 && cp <= 0x2A6DF) ||
[INFO] [stderr] 61 |             (cp >= 0x2A700 && cp <= 0x2B73F) ||
[INFO] [stderr] 62 |             (cp >= 0x2B740 && cp <= 0x2B81F) ||
[INFO] [stderr] 63 |             (cp >= 0x2B820 && cp <= 0x2CEAF) ||
[INFO] [stderr]  ...
[INFO] [stderr] 
[INFO] [stderr] warning: unneeded return statement
[INFO] [stderr]    --> src/tokenization.rs:198:9
[INFO] [stderr]     |
[INFO] [stderr] 198 |         return output_tokens;
[INFO] [stderr]     |         ^^^^^^^^^^^^^^^^^^^^^ help: remove `return` as shown: `output_tokens`
[INFO] [stderr]     |
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
[INFO] [stderr] 
[INFO] [stderr] warning: unneeded return statement
[INFO] [stderr]    --> src/tokenization.rs:241:9
[INFO] [stderr]     |
[INFO] [stderr] 241 |         return output_tokens;
[INFO] [stderr]     |         ^^^^^^^^^^^^^^^^^^^^^ help: remove `return` as shown: `output_tokens`
[INFO] [stderr]     |
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
[INFO] [stderr] 
[INFO] [stderr] warning: returning the result of a let binding from a block. Consider returning the expression directly.
[INFO] [stderr]   --> src/ffi.rs:18:5
[INFO] [stderr]    |
[INFO] [stderr] 18 |     error_ptr
[INFO] [stderr]    |     ^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = note: #[warn(clippy::let_and_return)] on by default
[INFO] [stderr] note: this expression can be directly returned
[INFO] [stderr]   --> src/ffi.rs:15:21
[INFO] [stderr]    |
[INFO] [stderr] 15 |       let error_ptr = ERROR_MSG.with(|error_msg|{
[INFO] [stderr]    |  _____________________^
[INFO] [stderr] 16 | |         return error_msg.borrow().as_ptr();
[INFO] [stderr] 17 | |     });
[INFO] [stderr]    | |______^
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#let_and_return
[INFO] [stderr] 
[INFO] [stderr] warning: unneeded return statement
[INFO] [stderr]   --> src/ffi.rs:16:9
[INFO] [stderr]    |
[INFO] [stderr] 16 |         return error_msg.borrow().as_ptr();
[INFO] [stderr]    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: remove `return` as shown: `error_msg.borrow().as_ptr()`
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
[INFO] [stderr] 
[INFO] [stderr] warning: redundant closure found
[INFO] [stderr]    --> src/tokenization.rs:130:23
[INFO] [stderr]     |
[INFO] [stderr] 130 |             .flat_map(|c|BasicTokenizer::_run_split_on_punc(c))
[INFO] [stderr]     |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: remove closure as shown: `BasicTokenizer::_run_split_on_punc`
[INFO] [stderr]     |
[INFO] [stderr]     = note: #[warn(clippy::redundant_closure)] on by default
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_closure
[INFO] [stderr] 
[INFO] [stderr] warning: length comparison to zero
[INFO] [stderr]    --> src/tokenization.rs:132:25
[INFO] [stderr]     |
[INFO] [stderr] 132 |             .filter(|s| s.len() > 0).collect();
[INFO] [stderr]     |                         ^^^^^^^^^^^ help: using `is_empty` is clearer and more explicit: `!s.is_empty()`
[INFO] [stderr]     |
[INFO] [stderr]     = note: #[warn(clippy::len_zero)] on by default
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#len_zero
[INFO] [stderr] 
[INFO] [stderr] warning: length comparison to zero
[INFO] [stderr]    --> src/tokenization.rs:185:20
[INFO] [stderr]     |
[INFO] [stderr] 185 |                 if cur_substr.len() == 0{
[INFO] [stderr]     |                    ^^^^^^^^^^^^^^^^^^^^^ help: using `is_empty` is clearer and more explicit: `cur_substr.is_empty()`
[INFO] [stderr]     |
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#len_zero
[INFO] [stderr] 
[INFO] [stderr] warning: using `clone` on a `Copy` type
[INFO] [stderr]    --> src/tokenization.rs:257:27
[INFO] [stderr]     |
[INFO] [stderr] 257 |     tokens.iter().map(|k| vocab.get(k).unwrap().clone()).collect()
[INFO] [stderr]     |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try dereferencing it: `*vocab.get(k).unwrap()`
[INFO] [stderr]     |
[INFO] [stderr]     = note: #[warn(clippy::clone_on_copy)] on by default
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
[INFO] [stderr] 
[INFO] [stderr] warning: very complex type used. Consider factoring parts into `type` definitions
[INFO] [stderr]    --> src/tokenization.rs:265:52
[INFO] [stderr]     |
[INFO] [stderr] 265 |       fn load_vocab<T: AsRef<str>>(vocab_file: T) -> Result<(IndexMap<String, usize>,
[INFO] [stderr]     |  ____________________________________________________^
[INFO] [stderr] 266 | |                                                            IndexMap<usize, String>), Box<Error>> {
[INFO] [stderr]     | |________________________________________________________________________________________________^
[INFO] [stderr]     |
[INFO] [stderr]     = note: #[warn(clippy::type_complexity)] on by default
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#type_complexity
[INFO] [stderr] 
[INFO] [stderr] warning: an inclusive range would be more readable
[INFO] [stderr]    --> src/tokenization.rs:365:22
[INFO] [stderr]     |
[INFO] [stderr] 365 |             for _ in 0..tokens_b.len()+1{
[INFO] [stderr]     |                      ^^^^^^^^^^^^^^^^^^^ help: use: `0..=tokens_b.len()`
[INFO] [stderr]     |
[INFO] [stderr]     = note: #[warn(clippy::range_plus_one)] on by default
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#range_plus_one
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:23:46
[INFO] [stderr]    |
[INFO] [stderr] 23 |     let vocab_file = unsafe { CStr::from_ptr(vocab_file)};
[INFO] [stderr]    |                                              ^^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = note: #[deny(clippy::not_unsafe_ptr_arg_deref)] on by default
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:43:27
[INFO] [stderr]    |
[INFO] [stderr] 43 |     unsafe {Box::from_raw(tokenizer)};
[INFO] [stderr]    |                           ^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:50:38
[INFO] [stderr]    |
[INFO] [stderr] 50 |     let text =unsafe{ CStr::from_ptr(text)}.to_string_lossy();
[INFO] [stderr]    |                                      ^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:56:14
[INFO] [stderr]    |
[INFO] [stderr] 56 |     unsafe {*output_len = ids.len() as _;}
[INFO] [stderr]    |              ^^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:64:44
[INFO] [stderr]    |
[INFO] [stderr] 64 |     let _ids = unsafe {Vec::from_raw_parts(ids_ptr,len as usize, len as usize)};
[INFO] [stderr]    |                                            ^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:72:40
[INFO] [stderr]    |
[INFO] [stderr] 72 |     let text_a =unsafe{ CStr::from_ptr(text_a)}.to_string_lossy();
[INFO] [stderr]    |                                        ^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:73:40
[INFO] [stderr]    |
[INFO] [stderr] 73 |     let text_b =unsafe{ CStr::from_ptr(text_b)}.to_string_lossy();
[INFO] [stderr]    |                                        ^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: aborting due to 7 previous errors
[INFO] [stderr] 
[INFO] [stderr] error: Could not compile `bert_tokenizer`.
[INFO] [stderr] warning: build failed, waiting for other jobs to finish...
[INFO] [stderr] warning: redundant closure found
[INFO] [stderr]    --> src/tokenization.rs:130:23
[INFO] [stderr]     |
[INFO] [stderr] 130 |             .flat_map(|c|BasicTokenizer::_run_split_on_punc(c))
[INFO] [stderr]     |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: remove closure as shown: `BasicTokenizer::_run_split_on_punc`
[INFO] [stderr]     |
[INFO] [stderr]     = note: #[warn(clippy::redundant_closure)] on by default
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_closure
[INFO] [stderr] 
[INFO] [stderr] warning: length comparison to zero
[INFO] [stderr]    --> src/tokenization.rs:132:25
[INFO] [stderr]     |
[INFO] [stderr] 132 |             .filter(|s| s.len() > 0).collect();
[INFO] [stderr]     |                         ^^^^^^^^^^^ help: using `is_empty` is clearer and more explicit: `!s.is_empty()`
[INFO] [stderr]     |
[INFO] [stderr]     = note: #[warn(clippy::len_zero)] on by default
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#len_zero
[INFO] [stderr] 
[INFO] [stderr] warning: length comparison to zero
[INFO] [stderr]    --> src/tokenization.rs:185:20
[INFO] [stderr]     |
[INFO] [stderr] 185 |                 if cur_substr.len() == 0{
[INFO] [stderr]     |                    ^^^^^^^^^^^^^^^^^^^^^ help: using `is_empty` is clearer and more explicit: `cur_substr.is_empty()`
[INFO] [stderr]     |
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#len_zero
[INFO] [stderr] 
[INFO] [stderr] warning: using `clone` on a `Copy` type
[INFO] [stderr]    --> src/tokenization.rs:257:27
[INFO] [stderr]     |
[INFO] [stderr] 257 |     tokens.iter().map(|k| vocab.get(k).unwrap().clone()).collect()
[INFO] [stderr]     |                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try dereferencing it: `*vocab.get(k).unwrap()`
[INFO] [stderr]     |
[INFO] [stderr]     = note: #[warn(clippy::clone_on_copy)] on by default
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
[INFO] [stderr] 
[INFO] [stderr] warning: very complex type used. Consider factoring parts into `type` definitions
[INFO] [stderr]    --> src/tokenization.rs:265:52
[INFO] [stderr]     |
[INFO] [stderr] 265 |       fn load_vocab<T: AsRef<str>>(vocab_file: T) -> Result<(IndexMap<String, usize>,
[INFO] [stderr]     |  ____________________________________________________^
[INFO] [stderr] 266 | |                                                            IndexMap<usize, String>), Box<Error>> {
[INFO] [stderr]     | |________________________________________________________________________________________________^
[INFO] [stderr]     |
[INFO] [stderr]     = note: #[warn(clippy::type_complexity)] on by default
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#type_complexity
[INFO] [stderr] 
[INFO] [stderr] warning: an inclusive range would be more readable
[INFO] [stderr]    --> src/tokenization.rs:365:22
[INFO] [stderr]     |
[INFO] [stderr] 365 |             for _ in 0..tokens_b.len()+1{
[INFO] [stderr]     |                      ^^^^^^^^^^^^^^^^^^^ help: use: `0..=tokens_b.len()`
[INFO] [stderr]     |
[INFO] [stderr]     = note: #[warn(clippy::range_plus_one)] on by default
[INFO] [stderr]     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#range_plus_one
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:23:46
[INFO] [stderr]    |
[INFO] [stderr] 23 |     let vocab_file = unsafe { CStr::from_ptr(vocab_file)};
[INFO] [stderr]    |                                              ^^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = note: #[deny(clippy::not_unsafe_ptr_arg_deref)] on by default
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:43:27
[INFO] [stderr]    |
[INFO] [stderr] 43 |     unsafe {Box::from_raw(tokenizer)};
[INFO] [stderr]    |                           ^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:50:38
[INFO] [stderr]    |
[INFO] [stderr] 50 |     let text =unsafe{ CStr::from_ptr(text)}.to_string_lossy();
[INFO] [stderr]    |                                      ^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:56:14
[INFO] [stderr]    |
[INFO] [stderr] 56 |     unsafe {*output_len = ids.len() as _;}
[INFO] [stderr]    |              ^^^^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:64:44
[INFO] [stderr]    |
[INFO] [stderr] 64 |     let _ids = unsafe {Vec::from_raw_parts(ids_ptr,len as usize, len as usize)};
[INFO] [stderr]    |                                            ^^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:72:40
[INFO] [stderr]    |
[INFO] [stderr] 72 |     let text_a =unsafe{ CStr::from_ptr(text_a)}.to_string_lossy();
[INFO] [stderr]    |                                        ^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: this public function dereferences a raw pointer but is not marked `unsafe`
[INFO] [stderr]   --> src/ffi.rs:73:40
[INFO] [stderr]    |
[INFO] [stderr] 73 |     let text_b =unsafe{ CStr::from_ptr(text_b)}.to_string_lossy();
[INFO] [stderr]    |                                        ^^^^^^
[INFO] [stderr]    |
[INFO] [stderr]    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#not_unsafe_ptr_arg_deref
[INFO] [stderr] 
[INFO] [stderr] error: aborting due to 7 previous errors
[INFO] [stderr] 
[INFO] [stderr] error: Could not compile `bert_tokenizer`.
[INFO] [stderr] 
[INFO] [stderr] To learn more, run the command again with --verbose.
[INFO] running `"docker" "inspect" "84d9769048f2598c35a317f223e237ce095d013f6cfb2280bfeb082a079e87ba"`
[INFO] running `"docker" "rm" "-f" "84d9769048f2598c35a317f223e237ce095d013f6cfb2280bfeb082a079e87ba"`
[INFO] [stdout] 84d9769048f2598c35a317f223e237ce095d013f6cfb2280bfeb082a079e87ba