Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 3760 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 8.2504. Min Range: -8.2504

Max Act: 8.2504. Min Act: -0.0000

Data Index: 443142 (The Pile)

Max Activating Token Index: 464

Text #1

Max Range: 8.2504. Min Range: -8.2504

Max Act: 7.6822. Min Act: -0.0000

Data Index: 477635 (The Pile)

Max Activating Token Index: 727

Text #2

Max Range: 8.2504. Min Range: -8.2504

Max Act: 7.2048. Min Act: -0.0000

Data Index: 1796887 (The Pile)

Max Activating Token Index: 469

Text #3

Max Range: 8.2504. Min Range: -8.2504

Max Act: 7.4056. Min Act: -0.0000

Data Index: 1086971 (The Pile)

Max Activating Token Index: 493

Text #4

Max Range: 8.2504. Min Range: -8.2504

Max Act: 6.9517. Min Act: -0.0000

Data Index: 1913420 (The Pile)

Max Activating Token Index: 963

Text #5

Max Range: 8.2504. Min Range: -8.2504

Max Act: 7.4269. Min Act: -0.0000

Data Index: 1641704 (The Pile)

Max Activating Token Index: 703

Text #6

Max Range: 8.2504. Min Range: -8.2504

Max Act: 7.2679. Min Act: -0.0000

Data Index: 1136680 (The Pile)

Max Activating Token Index: 891

Text #7

Max Range: 8.2504. Min Range: -8.2504

Max Act: 7.4112. Min Act: -0.0000

Data Index: 1204244 (The Pile)

Max Activating Token Index: 407

Text #8

Max Range: 8.2504. Min Range: -8.2504

Max Act: 6.5651. Min Act: -0.0000

Data Index: 1106702 (The Pile)

Max Activating Token Index: 980

Text #9

Max Range: 8.2504. Min Range: -8.2504

Max Act: 6.7725. Min Act: -0.0000

Data Index: 53128 (The Pile)

Max Activating Token Index: 801

Text #10

Max Range: 8.2504. Min Range: -8.2504

Max Act: 7.0824. Min Act: -0.0000

Data Index: 510354 (The Pile)

Max Activating Token Index: 959

Text #11

Max Range: 8.2504. Min Range: -8.2504

Max Act: 7.0721. Min Act: -0.0000

Data Index: 1913352 (The Pile)

Max Activating Token Index: 674

Text #12

Max Range: 8.2504. Min Range: -8.2504

Max Act: 6.8746. Min Act: -0.0000

Data Index: 1275297 (The Pile)

Max Activating Token Index: 311

Text #13

Max Range: 8.2504. Min Range: -8.2504

Max Act: 6.9029. Min Act: -0.0000

Data Index: 588300 (The Pile)

Max Activating Token Index: 770

Text #14

Max Range: 8.2504. Min Range: -8.2504

Max Act: 7.0365. Min Act: -0.0000

Data Index: 1267547 (The Pile)

Max Activating Token Index: 433

Text #15

Max Range: 8.2504. Min Range: -8.2504

Max Act: 7.1832. Min Act: -0.0000

Data Index: 1123624 (The Pile)

Max Activating Token Index: 812

Text #16

Max Range: 8.2504. Min Range: -8.2504

Max Act: 7.3554. Min Act: -0.0000

Data Index: 1656644 (The Pile)

Max Activating Token Index: 936

Text #17

Max Range: 8.2504. Min Range: -8.2504

Max Act: 6.7493. Min Act: -0.0000

Data Index: 480275 (The Pile)

Max Activating Token Index: 614

Text #18

Max Range: 8.2504. Min Range: -8.2504

Max Act: 6.1584. Min Act: -0.0000

Data Index: 117660 (The Pile)

Max Activating Token Index: 713

Text #19

Max Range: 8.2504. Min Range: -8.2504

Max Act: 6.5214. Min Act: -0.0000

Data Index: 1231524 (The Pile)

Max Activating Token Index: 251