Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4784 in Layer 9

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 3.5193. Min Range: -3.5193

Max Act: 3.5193. Min Act: -0.0000

Data Index: 268660 (The Pile)

Max Activating Token Index: 167

Text #1

Max Range: 3.5193. Min Range: -3.5193

Max Act: 3.2998. Min Act: -0.0000

Data Index: 379633 (The Pile)

Max Activating Token Index: 965

Text #2

Max Range: 3.5193. Min Range: -3.5193

Max Act: 3.0218. Min Act: -0.0000

Data Index: 1553708 (The Pile)

Max Activating Token Index: 208

Text #3

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.5796. Min Act: -0.0000

Data Index: 915855 (The Pile)

Max Activating Token Index: 453

Text #4

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.6452. Min Act: -0.0000

Data Index: 1110076 (The Pile)

Max Activating Token Index: 852

Text #5

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.4566. Min Act: -0.0000

Data Index: 830955 (The Pile)

Max Activating Token Index: 396

Text #6

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.4524. Min Act: -0.0000

Data Index: 988342 (The Pile)

Max Activating Token Index: 906

Text #7

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.6561. Min Act: -0.0000

Data Index: 166073 (The Pile)

Max Activating Token Index: 977

Text #8

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.2754. Min Act: -0.0000

Data Index: 81294 (The Pile)

Max Activating Token Index: 465

Text #9

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.2073. Min Act: -0.0000

Data Index: 751317 (The Pile)

Max Activating Token Index: 654

Text #10

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.0897. Min Act: -0.0000

Data Index: 870863 (The Pile)

Max Activating Token Index: 473

Text #11

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.2770. Min Act: -0.0000

Data Index: 355437 (The Pile)

Max Activating Token Index: 232

Text #12

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.1174. Min Act: -0.0000

Data Index: 909017 (The Pile)

Max Activating Token Index: 597

Text #13

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.0411. Min Act: -0.0000

Data Index: 1466752 (The Pile)

Max Activating Token Index: 889

Text #14

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.0825. Min Act: -0.0000

Data Index: 1603264 (The Pile)

Max Activating Token Index: 647

Text #15

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.2150. Min Act: -0.0000

Data Index: 1293783 (The Pile)

Max Activating Token Index: 958

Text #16

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.0573. Min Act: -0.0000

Data Index: 314881 (The Pile)

Max Activating Token Index: 17

Text #17

Max Range: 3.5193. Min Range: -3.5193

Max Act: 1.9538. Min Act: -0.0000

Data Index: 236779 (The Pile)

Max Activating Token Index: 88

Text #18

Max Range: 3.5193. Min Range: -3.5193

Max Act: 2.0290. Min Act: -0.0000

Data Index: 1477204 (The Pile)

Max Activating Token Index: 904

Text #19

Max Range: 3.5193. Min Range: -3.5193

Max Act: 1.9351. Min Act: -0.0000

Data Index: 969009 (The Pile)

Max Activating Token Index: 456