Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4342 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0227. Min Range: -1.0227

Max Act: 1.0076. Min Act: -0.0000

Data Index: 381900 (The Pile)

Max Activating Token Index: 749

Text #1

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.9500. Min Act: -0.0000

Data Index: 1388807 (The Pile)

Max Activating Token Index: 306

Text #2

Max Range: 1.0227. Min Range: -1.0227

Max Act: 1.0227. Min Act: -0.0000

Data Index: 1627396 (The Pile)

Max Activating Token Index: 657

Text #3

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.9349. Min Act: -0.0000

Data Index: 1430756 (The Pile)

Max Activating Token Index: 342

Text #4

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.8945. Min Act: -0.0000

Data Index: 1621986 (The Pile)

Max Activating Token Index: 887

Text #5

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.9141. Min Act: -0.0000

Data Index: 796098 (The Pile)

Max Activating Token Index: 339

Text #6

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.9460. Min Act: -0.0000

Data Index: 772633 (The Pile)

Max Activating Token Index: 47

Text #7

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.8700. Min Act: -0.0000

Data Index: 250139 (The Pile)

Max Activating Token Index: 673

Text #8

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.9261. Min Act: -0.0000

Data Index: 1614983 (The Pile)

Max Activating Token Index: 26

Text #9

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.9152. Min Act: -0.0000

Data Index: 1927293 (The Pile)

Max Activating Token Index: 828

Text #10

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.9105. Min Act: -0.0000

Data Index: 1301855 (The Pile)

Max Activating Token Index: 728

Text #11

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.8587. Min Act: -0.0000

Data Index: 225096 (The Pile)

Max Activating Token Index: 598

Text #12

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.8583. Min Act: -0.0000

Data Index: 82058 (The Pile)

Max Activating Token Index: 324

Text #13

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.8369. Min Act: -0.0000

Data Index: 1940744 (The Pile)

Max Activating Token Index: 991

Text #14

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.8522. Min Act: -0.0000

Data Index: 1746667 (The Pile)

Max Activating Token Index: 347

Text #15

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.8697. Min Act: -0.0000

Data Index: 640763 (The Pile)

Max Activating Token Index: 251

Text #16

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.8723. Min Act: -0.0000

Data Index: 1712767 (The Pile)

Max Activating Token Index: 639

Text #17

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.8943. Min Act: -0.0000

Data Index: 672805 (The Pile)

Max Activating Token Index: 308

Text #18

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.9218. Min Act: -0.0000

Data Index: 1912749 (The Pile)

Max Activating Token Index: 904

Text #19

Max Range: 1.0227. Min Range: -1.0227

Max Act: 0.8709. Min Act: -0.0000

Data Index: 1458903 (The Pile)

Max Activating Token Index: 202