Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 787 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.9578. Min Act: -0.0000

Data Index: 1452763 (The Pile)

Max Activating Token Index: 278

Text #1

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.8551. Min Act: -0.0000

Data Index: 251652 (The Pile)

Max Activating Token Index: 666

Text #2

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.8309. Min Act: -0.0000

Data Index: 385122 (The Pile)

Max Activating Token Index: 339

Text #3

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.7254. Min Act: -0.0000

Data Index: 1585901 (The Pile)

Max Activating Token Index: 467

Text #4

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.8016. Min Act: -0.0000

Data Index: 861581 (The Pile)

Max Activating Token Index: 633

Text #5

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.9095. Min Act: -0.0000

Data Index: 1391234 (The Pile)

Max Activating Token Index: 81

Text #6

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.7751. Min Act: -0.0000

Data Index: 1689034 (The Pile)

Max Activating Token Index: 635

Text #7

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.5495. Min Act: -0.0000

Data Index: 1944948 (The Pile)

Max Activating Token Index: 176

Text #8

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.6117. Min Act: -0.0000

Data Index: 1405640 (The Pile)

Max Activating Token Index: 247

Text #9

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.6767. Min Act: -0.0000

Data Index: 1035505 (The Pile)

Max Activating Token Index: 683

Text #10

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.6175. Min Act: -0.0000

Data Index: 1298180 (The Pile)

Max Activating Token Index: 73

Text #11

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.5789. Min Act: -0.0000

Data Index: 379253 (The Pile)

Max Activating Token Index: 631

Text #12

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.5645. Min Act: -0.0000

Data Index: 1267174 (The Pile)

Max Activating Token Index: 98

Text #13

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.5858. Min Act: -0.0000

Data Index: 1547031 (The Pile)

Max Activating Token Index: 636

Text #14

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.4612. Min Act: -0.0000

Data Index: 1278290 (The Pile)

Max Activating Token Index: 226

Text #15

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.6051. Min Act: -0.0000

Data Index: 1546892 (The Pile)

Max Activating Token Index: 500

Text #16

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.4237. Min Act: -0.0000

Data Index: 591319 (The Pile)

Max Activating Token Index: 747

Text #17

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.5112. Min Act: -0.0000

Data Index: 200758 (The Pile)

Max Activating Token Index: 395

Text #18

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.4521. Min Act: -0.0000

Data Index: 930093 (The Pile)

Max Activating Token Index: 529

Text #19

Max Range: 1.9578. Min Range: -1.9578

Max Act: 1.3610. Min Act: -0.0000

Data Index: 1888907 (The Pile)

Max Activating Token Index: 890