Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2059 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 5.3448. Min Range: -5.3448

Max Act: 5.3448. Min Act: -0.0000

Data Index: 1433340 (The Pile)

Max Activating Token Index: 389

Text #1

Max Range: 5.3448. Min Range: -5.3448

Max Act: 4.1952. Min Act: -0.0000

Data Index: 94279 (The Pile)

Max Activating Token Index: 764

Text #2

Max Range: 5.3448. Min Range: -5.3448

Max Act: 4.5991. Min Act: -0.0000

Data Index: 1525845 (The Pile)

Max Activating Token Index: 782

Text #3

Max Range: 5.3448. Min Range: -5.3448

Max Act: 4.3724. Min Act: -0.0000

Data Index: 105435 (The Pile)

Max Activating Token Index: 559

Text #4

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.8754. Min Act: -0.0000

Data Index: 1107212 (The Pile)

Max Activating Token Index: 574

Text #5

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.6830. Min Act: -0.0000

Data Index: 776467 (The Pile)

Max Activating Token Index: 7

Text #6

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.8027. Min Act: -0.0000

Data Index: 973485 (The Pile)

Max Activating Token Index: 823

Text #7

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.8605. Min Act: -0.0000

Data Index: 518131 (The Pile)

Max Activating Token Index: 38

Text #8

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.7833. Min Act: -0.0000

Data Index: 1929058 (The Pile)

Max Activating Token Index: 758

Text #9

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.5943. Min Act: -0.0000

Data Index: 564202 (The Pile)

Max Activating Token Index: 7

Text #10

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.2673. Min Act: -0.0000

Data Index: 1852962 (The Pile)

Max Activating Token Index: 188

Text #11

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.5487. Min Act: -0.0000

Data Index: 1612366 (The Pile)

Max Activating Token Index: 71

Text #12

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.2281. Min Act: -0.0000

Data Index: 1547392 (The Pile)

Max Activating Token Index: 402

Text #13

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.2291. Min Act: -0.0000

Data Index: 379003 (The Pile)

Max Activating Token Index: 15

Text #14

Max Range: 5.3448. Min Range: -5.3448

Max Act: 2.8725. Min Act: -0.0000

Data Index: 1282657 (The Pile)

Max Activating Token Index: 780

Text #15

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.1847. Min Act: -0.0000

Data Index: 1586769 (The Pile)

Max Activating Token Index: 528

Text #16

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.4411. Min Act: -0.0000

Data Index: 1913036 (The Pile)

Max Activating Token Index: 328

Text #17

Max Range: 5.3448. Min Range: -5.3448

Max Act: 2.8588. Min Act: -0.0000

Data Index: 167101 (The Pile)

Max Activating Token Index: 774

Text #18

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.1533. Min Act: -0.0000

Data Index: 410608 (The Pile)

Max Activating Token Index: 776

Text #19

Max Range: 5.3448. Min Range: -5.3448

Max Act: 3.1587. Min Act: -0.0000

Data Index: 1144483 (The Pile)

Max Activating Token Index: 317