Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2821 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 5.6467. Min Range: -5.6467

Max Act: 5.6467. Min Act: -0.0000

Data Index: 688625 (The Pile)

Max Activating Token Index: 530

Text #1

Max Range: 5.6467. Min Range: -5.6467

Max Act: 5.1257. Min Act: -0.0000

Data Index: 1437916 (The Pile)

Max Activating Token Index: 468

Text #2

Max Range: 5.6467. Min Range: -5.6467

Max Act: 5.2466. Min Act: -0.0000

Data Index: 1333135 (The Pile)

Max Activating Token Index: 51

Text #3

Max Range: 5.6467. Min Range: -5.6467

Max Act: 5.0563. Min Act: -0.0000

Data Index: 1800654 (The Pile)

Max Activating Token Index: 28

Text #4

Max Range: 5.6467. Min Range: -5.6467

Max Act: 5.0604. Min Act: -0.0000

Data Index: 1179077 (The Pile)

Max Activating Token Index: 874

Text #5

Max Range: 5.6467. Min Range: -5.6467

Max Act: 4.8568. Min Act: -0.0000

Data Index: 211098 (The Pile)

Max Activating Token Index: 844

Text #6

Max Range: 5.6467. Min Range: -5.6467

Max Act: 4.8828. Min Act: -0.0000

Data Index: 680277 (The Pile)

Max Activating Token Index: 640

Text #7

Max Range: 5.6467. Min Range: -5.6467

Max Act: 5.2034. Min Act: -0.0000

Data Index: 1524633 (The Pile)

Max Activating Token Index: 340

Text #8

Max Range: 5.6467. Min Range: -5.6467

Max Act: 5.1577. Min Act: -0.0000

Data Index: 1404562 (The Pile)

Max Activating Token Index: 663

Text #9

Max Range: 5.6467. Min Range: -5.6467

Max Act: 5.1703. Min Act: -0.0000

Data Index: 1431512 (The Pile)

Max Activating Token Index: 830

Text #10

Max Range: 5.6467. Min Range: -5.6467

Max Act: 5.1151. Min Act: -0.0000

Data Index: 971504 (The Pile)

Max Activating Token Index: 847

Text #11

Max Range: 5.6467. Min Range: -5.6467

Max Act: 4.9523. Min Act: -0.0000

Data Index: 1921905 (The Pile)

Max Activating Token Index: 534

Text #12

Max Range: 5.6467. Min Range: -5.6467

Max Act: 4.9326. Min Act: -0.0000

Data Index: 209446 (The Pile)

Max Activating Token Index: 809

Text #13

Max Range: 5.6467. Min Range: -5.6467

Max Act: 4.6756. Min Act: -0.0000

Data Index: 787269 (The Pile)

Max Activating Token Index: 568

Text #14

Max Range: 5.6467. Min Range: -5.6467

Max Act: 5.1628. Min Act: -0.0001

Data Index: 1120713 (The Pile)

Max Activating Token Index: 912

Text #15

Max Range: 5.6467. Min Range: -5.6467

Max Act: 4.9919. Min Act: -0.0000

Data Index: 911174 (The Pile)

Max Activating Token Index: 319

Text #16

Max Range: 5.6467. Min Range: -5.6467

Max Act: 4.9458. Min Act: -0.0000

Data Index: 1231818 (The Pile)

Max Activating Token Index: 495

Text #17

Max Range: 5.6467. Min Range: -5.6467

Max Act: 5.0107. Min Act: -0.0000

Data Index: 816695 (The Pile)

Max Activating Token Index: 625

Text #18

Max Range: 5.6467. Min Range: -5.6467

Max Act: 4.9074. Min Act: -0.0000

Data Index: 1192141 (The Pile)

Max Activating Token Index: 493

Text #19

Max Range: 5.6467. Min Range: -5.6467

Max Act: 4.9348. Min Act: -0.0000

Data Index: 764120 (The Pile)

Max Activating Token Index: 325