Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2206 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.8930. Min Act: -0.0000

Data Index: 1773569 (The Pile)

Max Activating Token Index: 781

Text #1

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.3792. Min Act: -0.0000

Data Index: 1158840 (The Pile)

Max Activating Token Index: 728

Text #2

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.5363. Min Act: -0.0000

Data Index: 428846 (The Pile)

Max Activating Token Index: 161

Text #3

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.5433. Min Act: -0.0000

Data Index: 1747985 (The Pile)

Max Activating Token Index: 341

Text #4

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.4287. Min Act: -0.0000

Data Index: 1252172 (The Pile)

Max Activating Token Index: 886

Text #5

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.4450. Min Act: -0.0000

Data Index: 1702844 (The Pile)

Max Activating Token Index: 120

Text #6

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.4281. Min Act: -0.0000

Data Index: 655709 (The Pile)

Max Activating Token Index: 292

Text #7

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.3514. Min Act: -0.0000

Data Index: 1764996 (The Pile)

Max Activating Token Index: 287

Text #8

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.3168. Min Act: -0.0000

Data Index: 1873583 (The Pile)

Max Activating Token Index: 268

Text #9

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.3474. Min Act: -0.0000

Data Index: 1047706 (The Pile)

Max Activating Token Index: 39

Text #10

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.3005. Min Act: -0.0000

Data Index: 1724200 (The Pile)

Max Activating Token Index: 36

Text #11

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.2874. Min Act: -0.0000

Data Index: 810695 (The Pile)

Max Activating Token Index: 340

Text #12

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.3253. Min Act: -0.0000

Data Index: 1042322 (The Pile)

Max Activating Token Index: 751

Text #13

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.1051. Min Act: -0.0000

Data Index: 1702004 (The Pile)

Max Activating Token Index: 708

Text #14

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.2320. Min Act: -0.0000

Data Index: 313953 (The Pile)

Max Activating Token Index: 813

Text #15

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.2634. Min Act: -0.0000

Data Index: 1025773 (The Pile)

Max Activating Token Index: 79

Text #16

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.3271. Min Act: -0.0000

Data Index: 148746 (The Pile)

Max Activating Token Index: 783

Text #17

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.2736. Min Act: -0.0000

Data Index: 476254 (The Pile)

Max Activating Token Index: 1018

Text #18

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.1562. Min Act: -0.0000

Data Index: 1736849 (The Pile)

Max Activating Token Index: 185

Text #19

Max Range: 1.8930. Min Range: -1.8930

Max Act: 1.1825. Min Act: -0.0000

Data Index: 1695691 (The Pile)

Max Activating Token Index: 467