Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1681 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.9256. Min Act: -0.0000

Data Index: 582213 (The Pile)

Max Activating Token Index: 644

Text #1

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.7585. Min Act: -0.0000

Data Index: 258099 (The Pile)

Max Activating Token Index: 494

Text #2

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.7385. Min Act: -0.0000

Data Index: 1003481 (The Pile)

Max Activating Token Index: 250

Text #3

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6615. Min Act: -0.0000

Data Index: 652671 (The Pile)

Max Activating Token Index: 674

Text #4

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6393. Min Act: -0.0000

Data Index: 1102952 (The Pile)

Max Activating Token Index: 300

Text #5

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.7117. Min Act: -0.0000

Data Index: 1058445 (The Pile)

Max Activating Token Index: 882

Text #6

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6014. Min Act: -0.0000

Data Index: 803101 (The Pile)

Max Activating Token Index: 759

Text #7

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.7226. Min Act: -0.0000

Data Index: 1899962 (The Pile)

Max Activating Token Index: 660

Text #8

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6521. Min Act: -0.0000

Data Index: 1162958 (The Pile)

Max Activating Token Index: 192

Text #9

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6510. Min Act: -0.0000

Data Index: 599324 (The Pile)

Max Activating Token Index: 300

Text #10

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6733. Min Act: -0.0000

Data Index: 321135 (The Pile)

Max Activating Token Index: 438

Text #11

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6458. Min Act: -0.0000

Data Index: 1740298 (The Pile)

Max Activating Token Index: 709

Text #12

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6289. Min Act: -0.0000

Data Index: 1174153 (The Pile)

Max Activating Token Index: 363

Text #13

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6172. Min Act: -0.0000

Data Index: 810465 (The Pile)

Max Activating Token Index: 283

Text #14

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.7127. Min Act: -0.0000

Data Index: 461020 (The Pile)

Max Activating Token Index: 655

Text #15

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6286. Min Act: -0.0000

Data Index: 1274556 (The Pile)

Max Activating Token Index: 611

Text #16

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6285. Min Act: -0.0000

Data Index: 311475 (The Pile)

Max Activating Token Index: 472

Text #17

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6378. Min Act: -0.0000

Data Index: 164029 (The Pile)

Max Activating Token Index: 464

Text #18

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6046. Min Act: -0.0000

Data Index: 912029 (The Pile)

Max Activating Token Index: 164

Text #19

Max Range: 0.9256. Min Range: -0.9256

Max Act: 0.6471. Min Act: -0.0000

Data Index: 885300 (The Pile)

Max Activating Token Index: 388