Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1737 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.5640. Min Act: -0.0000

Data Index: 1191064 (The Pile)

Max Activating Token Index: 360

Text #1

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.9382. Min Act: -0.0000

Data Index: 1690914 (The Pile)

Max Activating Token Index: 1015

Text #2

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.5651. Min Act: -0.0000

Data Index: 1573863 (The Pile)

Max Activating Token Index: 852

Text #3

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.5205. Min Act: -0.0000

Data Index: 1098672 (The Pile)

Max Activating Token Index: 483

Text #4

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.5161. Min Act: -0.0000

Data Index: 1619063 (The Pile)

Max Activating Token Index: 626

Text #5

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.2771. Min Act: -0.0000

Data Index: 863346 (The Pile)

Max Activating Token Index: 910

Text #6

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.5727. Min Act: -0.0000

Data Index: 1129397 (The Pile)

Max Activating Token Index: 631

Text #7

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.3769. Min Act: -0.0000

Data Index: 994306 (The Pile)

Max Activating Token Index: 158

Text #8

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.4441. Min Act: -0.0000

Data Index: 541546 (The Pile)

Max Activating Token Index: 268

Text #9

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.4561. Min Act: -0.0000

Data Index: 1762675 (The Pile)

Max Activating Token Index: 111

Text #10

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.4514. Min Act: -0.0000

Data Index: 673411 (The Pile)

Max Activating Token Index: 453

Text #11

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.3208. Min Act: -0.0000

Data Index: 1668821 (The Pile)

Max Activating Token Index: 472

Text #12

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.2817. Min Act: -0.0000

Data Index: 1345390 (The Pile)

Max Activating Token Index: 17

Text #13

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.1770. Min Act: -0.0000

Data Index: 238413 (The Pile)

Max Activating Token Index: 68

Text #14

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.3117. Min Act: -0.0000

Data Index: 827200 (The Pile)

Max Activating Token Index: 84

Text #15

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.3163. Min Act: -0.0000

Data Index: 11237 (The Pile)

Max Activating Token Index: 869

Text #16

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.2570. Min Act: -0.0000

Data Index: 1696577 (The Pile)

Max Activating Token Index: 440

Text #17

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.2694. Min Act: -0.0000

Data Index: 590664 (The Pile)

Max Activating Token Index: 593

Text #18

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.1899. Min Act: -0.0000

Data Index: 1596311 (The Pile)

Max Activating Token Index: 342

Text #19

Max Range: 1.9382. Min Range: -1.9382

Max Act: 1.1926. Min Act: -0.0000

Data Index: 1165289 (The Pile)

Max Activating Token Index: 827