Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4693 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.9497. Min Act: 0.0000

Data Index: 1626085 (The Pile)

Max Activating Token Index: 404

Text #1

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.7006. Min Act: -0.0000

Data Index: 584809 (The Pile)

Max Activating Token Index: 578

Text #2

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.6606. Min Act: -0.0000

Data Index: 1396078 (The Pile)

Max Activating Token Index: 236

Text #3

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.5471. Min Act: -0.0000

Data Index: 1261659 (The Pile)

Max Activating Token Index: 101

Text #4

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.7093. Min Act: -0.0000

Data Index: 1303541 (The Pile)

Max Activating Token Index: 703

Text #5

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.6154. Min Act: 0.0000

Data Index: 1881203 (The Pile)

Max Activating Token Index: 732

Text #6

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.4919. Min Act: -0.0000

Data Index: 92335 (The Pile)

Max Activating Token Index: 793

Text #7

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.5572. Min Act: -0.0000

Data Index: 196323 (The Pile)

Max Activating Token Index: 175

Text #8

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.4034. Min Act: 0.0000

Data Index: 1366505 (The Pile)

Max Activating Token Index: 693

Text #9

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.6101. Min Act: -0.0000

Data Index: 1617871 (The Pile)

Max Activating Token Index: 512

Text #10

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.5200. Min Act: -0.0000

Data Index: 1960338 (The Pile)

Max Activating Token Index: 297

Text #11

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.4908. Min Act: -0.0000

Data Index: 1496419 (The Pile)

Max Activating Token Index: 500

Text #12

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.4910. Min Act: -0.0000

Data Index: 360314 (The Pile)

Max Activating Token Index: 185

Text #13

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.4678. Min Act: -0.0000

Data Index: 902367 (The Pile)

Max Activating Token Index: 926

Text #14

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.4673. Min Act: -0.0000

Data Index: 758999 (The Pile)

Max Activating Token Index: 967

Text #15

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.5287. Min Act: -0.0000

Data Index: 1309388 (The Pile)

Max Activating Token Index: 696

Text #16

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.4097. Min Act: -0.0000

Data Index: 1884057 (The Pile)

Max Activating Token Index: 408

Text #17

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.3414. Min Act: 0.0000

Data Index: 524266 (The Pile)

Max Activating Token Index: 782

Text #18

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.4047. Min Act: -0.0000

Data Index: 727761 (The Pile)

Max Activating Token Index: 389

Text #19

Max Range: 1.9497. Min Range: -1.9497

Max Act: 1.4024. Min Act: -0.0000

Data Index: 177612 (The Pile)

Max Activating Token Index: 142