Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 577 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 4.0007. Min Range: -4.0007

Max Act: 4.0007. Min Act: -0.0000

Data Index: 59538 (The Pile)

Max Activating Token Index: 469

Text #1

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.7150. Min Act: -0.0000

Data Index: 1337206 (The Pile)

Max Activating Token Index: 333

Text #2

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.6343. Min Act: -0.0000

Data Index: 98558 (The Pile)

Max Activating Token Index: 439

Text #3

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.8818. Min Act: -0.0000

Data Index: 1454718 (The Pile)

Max Activating Token Index: 270

Text #4

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.3136. Min Act: -0.0000

Data Index: 1560251 (The Pile)

Max Activating Token Index: 634

Text #5

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.6677. Min Act: -0.0000

Data Index: 302434 (The Pile)

Max Activating Token Index: 561

Text #6

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.2931. Min Act: -0.0000

Data Index: 1877866 (The Pile)

Max Activating Token Index: 476

Text #7

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.4407. Min Act: -0.0000

Data Index: 754430 (The Pile)

Max Activating Token Index: 128

Text #8

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.4091. Min Act: -0.0000

Data Index: 83260 (The Pile)

Max Activating Token Index: 436

Text #9

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.3522. Min Act: -0.0000

Data Index: 1438273 (The Pile)

Max Activating Token Index: 438

Text #10

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.3521. Min Act: -0.0000

Data Index: 442723 (The Pile)

Max Activating Token Index: 736

Text #11

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.3511. Min Act: -0.0000

Data Index: 336702 (The Pile)

Max Activating Token Index: 316

Text #12

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.1996. Min Act: -0.0000

Data Index: 1736151 (The Pile)

Max Activating Token Index: 320

Text #13

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.0845. Min Act: -0.0000

Data Index: 816309 (The Pile)

Max Activating Token Index: 778

Text #14

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.3323. Min Act: -0.0000

Data Index: 232999 (The Pile)

Max Activating Token Index: 350

Text #15

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.4680. Min Act: -0.0000

Data Index: 1734072 (The Pile)

Max Activating Token Index: 372

Text #16

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.2974. Min Act: -0.0000

Data Index: 1677770 (The Pile)

Max Activating Token Index: 617

Text #17

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.6497. Min Act: -0.0000

Data Index: 125839 (The Pile)

Max Activating Token Index: 441

Text #18

Max Range: 4.0007. Min Range: -4.0007

Max Act: 2.3196. Min Act: -0.0000

Data Index: 1586675 (The Pile)

Max Activating Token Index: 731

Text #19

Max Range: 4.0007. Min Range: -4.0007

Max Act: 3.6286. Min Act: -0.0000

Data Index: 764215 (The Pile)

Max Activating Token Index: 395