Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 686 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.0764. Min Range: -1.0764

Max Act: 1.0011. Min Act: -0.0000

Data Index: 665678 (The Pile)

Max Activating Token Index: 252

Text #1

Max Range: 1.0764. Min Range: -1.0764

Max Act: 1.0764. Min Act: -0.0000

Data Index: 221506 (The Pile)

Max Activating Token Index: 594

Text #2

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.9325. Min Act: -0.0000

Data Index: 1229534 (The Pile)

Max Activating Token Index: 370

Text #3

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.9197. Min Act: -0.0000

Data Index: 1038256 (The Pile)

Max Activating Token Index: 560

Text #4

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.9309. Min Act: -0.0000

Data Index: 1023214 (The Pile)

Max Activating Token Index: 399

Text #5

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.9091. Min Act: -0.0000

Data Index: 96757 (The Pile)

Max Activating Token Index: 265

Text #6

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.8467. Min Act: -0.0000

Data Index: 1896830 (The Pile)

Max Activating Token Index: 511

Text #7

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.9014. Min Act: -0.0000

Data Index: 865094 (The Pile)

Max Activating Token Index: 568

Text #8

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.8568. Min Act: -0.0000

Data Index: 1689924 (The Pile)

Max Activating Token Index: 419

Text #9

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.8088. Min Act: -0.0000

Data Index: 1611279 (The Pile)

Max Activating Token Index: 369

Text #10

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.7637. Min Act: -0.0000

Data Index: 1232752 (The Pile)

Max Activating Token Index: 768

Text #11

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.7926. Min Act: -0.0000

Data Index: 1408999 (The Pile)

Max Activating Token Index: 578

Text #12

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.7179. Min Act: -0.0000

Data Index: 1629983 (The Pile)

Max Activating Token Index: 551

Text #13

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.8697. Min Act: -0.0000

Data Index: 621298 (The Pile)

Max Activating Token Index: 322

Text #14

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.8570. Min Act: -0.0000

Data Index: 1731433 (The Pile)

Max Activating Token Index: 747

Text #15

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.7849. Min Act: -0.0000

Data Index: 79972 (The Pile)

Max Activating Token Index: 259

Text #16

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.8216. Min Act: -0.0000

Data Index: 170001 (The Pile)

Max Activating Token Index: 548

Text #17

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.7885. Min Act: -0.0000

Data Index: 1251445 (The Pile)

Max Activating Token Index: 515

Text #18

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.8133. Min Act: -0.0000

Data Index: 1288366 (The Pile)

Max Activating Token Index: 253

Text #19

Max Range: 1.0764. Min Range: -1.0764

Max Act: 0.8073. Min Act: -0.0000

Data Index: 846731 (The Pile)

Max Activating Token Index: 360