Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3358 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.9562. Min Act: -0.0000

Data Index: 1125966 (The Pile)

Max Activating Token Index: 453

Text #1

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.8715. Min Act: -0.0000

Data Index: 836653 (The Pile)

Max Activating Token Index: 843

Text #2

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.8633. Min Act: -0.0000

Data Index: 1901932 (The Pile)

Max Activating Token Index: 846

Text #3

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7877. Min Act: -0.0000

Data Index: 1352634 (The Pile)

Max Activating Token Index: 132

Text #4

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7410. Min Act: -0.0000

Data Index: 745542 (The Pile)

Max Activating Token Index: 463

Text #5

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.8000. Min Act: -0.0000

Data Index: 1195487 (The Pile)

Max Activating Token Index: 684

Text #6

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.8251. Min Act: -0.0000

Data Index: 1048049 (The Pile)

Max Activating Token Index: 625

Text #7

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7707. Min Act: -0.0000

Data Index: 809249 (The Pile)

Max Activating Token Index: 696

Text #8

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7603. Min Act: -0.0000

Data Index: 342125 (The Pile)

Max Activating Token Index: 835

Text #9

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7094. Min Act: -0.0000

Data Index: 1418152 (The Pile)

Max Activating Token Index: 524

Text #10

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7466. Min Act: -0.0000

Data Index: 1341311 (The Pile)

Max Activating Token Index: 514

Text #11

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.6888. Min Act: -0.0000

Data Index: 260162 (The Pile)

Max Activating Token Index: 546

Text #12

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7411. Min Act: -0.0000

Data Index: 1060502 (The Pile)

Max Activating Token Index: 536

Text #13

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7409. Min Act: -0.0000

Data Index: 582708 (The Pile)

Max Activating Token Index: 499

Text #14

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7581. Min Act: -0.0000

Data Index: 499367 (The Pile)

Max Activating Token Index: 578

Text #15

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7310. Min Act: -0.0000

Data Index: 1796742 (The Pile)

Max Activating Token Index: 381

Text #16

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7319. Min Act: -0.0000

Data Index: 1691604 (The Pile)

Max Activating Token Index: 998

Text #17

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7005. Min Act: -0.0000

Data Index: 484332 (The Pile)

Max Activating Token Index: 762

Text #18

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7175. Min Act: -0.0000

Data Index: 563978 (The Pile)

Max Activating Token Index: 526

Text #19

Max Range: 0.9562. Min Range: -0.9562

Max Act: 0.7378. Min Act: -0.0000

Data Index: 1120837 (The Pile)

Max Activating Token Index: 387