Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1335 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.6265. Min Act: -0.0000

Data Index: 1371798 (The Pile)

Max Activating Token Index: 159

Text #1

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.5824. Min Act: -0.0000

Data Index: 1770500 (The Pile)

Max Activating Token Index: 658

Text #2

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.4920. Min Act: -0.0000

Data Index: 1212868 (The Pile)

Max Activating Token Index: 537

Text #3

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.5564. Min Act: -0.0000

Data Index: 683076 (The Pile)

Max Activating Token Index: 55

Text #4

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.4783. Min Act: -0.0000

Data Index: 1701011 (The Pile)

Max Activating Token Index: 625

Text #5

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.5378. Min Act: -0.0000

Data Index: 1092374 (The Pile)

Max Activating Token Index: 128

Text #6

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.5246. Min Act: -0.0000

Data Index: 1693353 (The Pile)

Max Activating Token Index: 713

Text #7

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.5071. Min Act: -0.0000

Data Index: 1375386 (The Pile)

Max Activating Token Index: 928

Text #8

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.5401. Min Act: -0.0000

Data Index: 253501 (The Pile)

Max Activating Token Index: 645

Text #9

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.4391. Min Act: -0.0000

Data Index: 651402 (The Pile)

Max Activating Token Index: 512

Text #10

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.4701. Min Act: -0.0000

Data Index: 1198695 (The Pile)

Max Activating Token Index: 711

Text #11

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.4300. Min Act: -0.0000

Data Index: 196085 (The Pile)

Max Activating Token Index: 725

Text #12

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.5280. Min Act: -0.0000

Data Index: 1863059 (The Pile)

Max Activating Token Index: 249

Text #13

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.4748. Min Act: -0.0000

Data Index: 207350 (The Pile)

Max Activating Token Index: 965

Text #14

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.4663. Min Act: -0.0000

Data Index: 1400119 (The Pile)

Max Activating Token Index: 130

Text #15

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.4557. Min Act: -0.0000

Data Index: 600886 (The Pile)

Max Activating Token Index: 130

Text #16

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.5022. Min Act: -0.0000

Data Index: 848106 (The Pile)

Max Activating Token Index: 544

Text #17

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.3879. Min Act: -0.0000

Data Index: 631145 (The Pile)

Max Activating Token Index: 1001

Text #18

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.5114. Min Act: -0.0000

Data Index: 1035243 (The Pile)

Max Activating Token Index: 564

Text #19

Max Range: 0.6265. Min Range: -0.6265

Max Act: 0.4378. Min Act: -0.0000

Data Index: 522599 (The Pile)

Max Activating Token Index: 862