Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3275 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.2582. Min Range: -1.2582

Max Act: 1.2582. Min Act: -0.0000

Data Index: 1256215 (The Pile)

Max Activating Token Index: 977

Text #1

Max Range: 1.2582. Min Range: -1.2582

Max Act: 1.1821. Min Act: -0.0000

Data Index: 1383057 (The Pile)

Max Activating Token Index: 990

Text #2

Max Range: 1.2582. Min Range: -1.2582

Max Act: 1.1548. Min Act: -0.0000

Data Index: 556994 (The Pile)

Max Activating Token Index: 374

Text #3

Max Range: 1.2582. Min Range: -1.2582

Max Act: 1.1296. Min Act: -0.0000

Data Index: 926449 (The Pile)

Max Activating Token Index: 515

Text #4

Max Range: 1.2582. Min Range: -1.2582

Max Act: 1.1422. Min Act: -0.0000

Data Index: 981865 (The Pile)

Max Activating Token Index: 400

Text #5

Max Range: 1.2582. Min Range: -1.2582

Max Act: 1.0455. Min Act: -0.0000

Data Index: 1042680 (The Pile)

Max Activating Token Index: 309

Text #6

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9957. Min Act: -0.0000

Data Index: 483678 (The Pile)

Max Activating Token Index: 696

Text #7

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9869. Min Act: -0.0000

Data Index: 1146098 (The Pile)

Max Activating Token Index: 97

Text #8

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9823. Min Act: -0.0000

Data Index: 1049408 (The Pile)

Max Activating Token Index: 720

Text #9

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9828. Min Act: -0.0000

Data Index: 495640 (The Pile)

Max Activating Token Index: 359

Text #10

Max Range: 1.2582. Min Range: -1.2582

Max Act: 1.0011. Min Act: -0.0000

Data Index: 14820 (The Pile)

Max Activating Token Index: 732

Text #11

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9559. Min Act: -0.0000

Data Index: 516586 (The Pile)

Max Activating Token Index: 889

Text #12

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9254. Min Act: -0.0000

Data Index: 883511 (The Pile)

Max Activating Token Index: 364

Text #13

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9493. Min Act: -0.0000

Data Index: 1079865 (The Pile)

Max Activating Token Index: 885

Text #14

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9763. Min Act: -0.0000

Data Index: 551751 (The Pile)

Max Activating Token Index: 896

Text #15

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9463. Min Act: -0.0000

Data Index: 218735 (The Pile)

Max Activating Token Index: 365

Text #16

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9334. Min Act: -0.0000

Data Index: 1298316 (The Pile)

Max Activating Token Index: 860

Text #17

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9091. Min Act: -0.0000

Data Index: 427682 (The Pile)

Max Activating Token Index: 1005

Text #18

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9876. Min Act: -0.0000

Data Index: 774490 (The Pile)

Max Activating Token Index: 624

Text #19

Max Range: 1.2582. Min Range: -1.2582

Max Act: 0.9623. Min Act: -0.0000

Data Index: 1633500 (The Pile)

Max Activating Token Index: 1017