Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2865 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.9452. Min Act: -0.0000

Data Index: 971349 (The Pile)

Max Activating Token Index: 53

Text #1

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.9070. Min Act: -0.0000

Data Index: 404417 (The Pile)

Max Activating Token Index: 535

Text #2

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.8441. Min Act: -0.0000

Data Index: 583772 (The Pile)

Max Activating Token Index: 626

Text #3

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7832. Min Act: -0.0000

Data Index: 246988 (The Pile)

Max Activating Token Index: 99

Text #4

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7784. Min Act: -0.0000

Data Index: 601134 (The Pile)

Max Activating Token Index: 520

Text #5

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7050. Min Act: -0.0000

Data Index: 1927307 (The Pile)

Max Activating Token Index: 1002

Text #6

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7591. Min Act: -0.0000

Data Index: 892991 (The Pile)

Max Activating Token Index: 133

Text #7

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7556. Min Act: -0.0000

Data Index: 1779297 (The Pile)

Max Activating Token Index: 508

Text #8

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7973. Min Act: -0.0000

Data Index: 1218282 (The Pile)

Max Activating Token Index: 942

Text #9

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7694. Min Act: -0.0000

Data Index: 1311597 (The Pile)

Max Activating Token Index: 966

Text #10

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7365. Min Act: -0.0000

Data Index: 594243 (The Pile)

Max Activating Token Index: 636

Text #11

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7633. Min Act: -0.0000

Data Index: 1329953 (The Pile)

Max Activating Token Index: 770

Text #12

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7466. Min Act: -0.0000

Data Index: 70569 (The Pile)

Max Activating Token Index: 104

Text #13

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7182. Min Act: -0.0000

Data Index: 960114 (The Pile)

Max Activating Token Index: 109

Text #14

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7626. Min Act: -0.0000

Data Index: 384138 (The Pile)

Max Activating Token Index: 56

Text #15

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7603. Min Act: -0.0000

Data Index: 862781 (The Pile)

Max Activating Token Index: 119

Text #16

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.6723. Min Act: -0.0000

Data Index: 412733 (The Pile)

Max Activating Token Index: 145

Text #17

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7072. Min Act: -0.0000

Data Index: 44270 (The Pile)

Max Activating Token Index: 691

Text #18

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.7149. Min Act: -0.0000

Data Index: 458509 (The Pile)

Max Activating Token Index: 43

Text #19

Max Range: 0.9452. Min Range: -0.9452

Max Act: 0.6636. Min Act: -0.0000

Data Index: 1490687 (The Pile)

Max Activating Token Index: 585