Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2905 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.5195. Min Act: -0.0000

Data Index: 81558 (The Pile)

Max Activating Token Index: 146

Text #1

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.4122. Min Act: -0.0000

Data Index: 461081 (The Pile)

Max Activating Token Index: 892

Text #2

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.4019. Min Act: -0.0000

Data Index: 1998031 (The Pile)

Max Activating Token Index: 326

Text #3

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.4116. Min Act: -0.0000

Data Index: 146600 (The Pile)

Max Activating Token Index: 375

Text #4

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.4022. Min Act: -0.0000

Data Index: 1933410 (The Pile)

Max Activating Token Index: 283

Text #5

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3719. Min Act: -0.0000

Data Index: 491150 (The Pile)

Max Activating Token Index: 542

Text #6

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3735. Min Act: -0.0000

Data Index: 844846 (The Pile)

Max Activating Token Index: 965

Text #7

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3375. Min Act: -0.0000

Data Index: 1900589 (The Pile)

Max Activating Token Index: 71

Text #8

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3704. Min Act: -0.0000

Data Index: 1902962 (The Pile)

Max Activating Token Index: 345

Text #9

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3408. Min Act: -0.0000

Data Index: 1205096 (The Pile)

Max Activating Token Index: 217

Text #10

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3718. Min Act: -0.0000

Data Index: 711495 (The Pile)

Max Activating Token Index: 489

Text #11

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3601. Min Act: -0.0000

Data Index: 1191294 (The Pile)

Max Activating Token Index: 498

Text #12

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3257. Min Act: -0.0000

Data Index: 967593 (The Pile)

Max Activating Token Index: 574

Text #13

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3356. Min Act: -0.0000

Data Index: 1103211 (The Pile)

Max Activating Token Index: 689

Text #14

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3087. Min Act: -0.0000

Data Index: 208175 (The Pile)

Max Activating Token Index: 765

Text #15

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3279. Min Act: -0.0000

Data Index: 321189 (The Pile)

Max Activating Token Index: 277

Text #16

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3480. Min Act: -0.0000

Data Index: 99793 (The Pile)

Max Activating Token Index: 53

Text #17

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3272. Min Act: -0.0000

Data Index: 524196 (The Pile)

Max Activating Token Index: 270

Text #18

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3238. Min Act: -0.0000

Data Index: 670712 (The Pile)

Max Activating Token Index: 244

Text #19

Max Range: 0.5195. Min Range: -0.5195

Max Act: 0.3238. Min Act: -0.0000

Data Index: 1372247 (The Pile)

Max Activating Token Index: 181