Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 1953 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.4748. Min Act: -0.0000

Data Index: 1387465 (The Pile)

Max Activating Token Index: 844

Text #1

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.1651. Min Act: -0.0000

Data Index: 1946065 (The Pile)

Max Activating Token Index: 552

Text #2

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.1074. Min Act: -0.0000

Data Index: 1139553 (The Pile)

Max Activating Token Index: 607

Text #3

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.1165. Min Act: -0.0000

Data Index: 227102 (The Pile)

Max Activating Token Index: 965

Text #4

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.1677. Min Act: -0.0000

Data Index: 805605 (The Pile)

Max Activating Token Index: 776

Text #5

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.0972. Min Act: -0.0000

Data Index: 482104 (The Pile)

Max Activating Token Index: 508

Text #6

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.1179. Min Act: -0.0000

Data Index: 1917867 (The Pile)

Max Activating Token Index: 634

Text #7

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.0149. Min Act: -0.0000

Data Index: 825919 (The Pile)

Max Activating Token Index: 306

Text #8

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.0631. Min Act: -0.0000

Data Index: 1142833 (The Pile)

Max Activating Token Index: 869

Text #9

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.0233. Min Act: -0.0000

Data Index: 839548 (The Pile)

Max Activating Token Index: 496

Text #10

Max Range: 1.4748. Min Range: -1.4748

Max Act: 0.9980. Min Act: -0.0000

Data Index: 619465 (The Pile)

Max Activating Token Index: 526

Text #11

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.1063. Min Act: -0.0000

Data Index: 889794 (The Pile)

Max Activating Token Index: 686

Text #12

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.0247. Min Act: -0.0000

Data Index: 103157 (The Pile)

Max Activating Token Index: 296

Text #13

Max Range: 1.4748. Min Range: -1.4748

Max Act: 0.9572. Min Act: -0.0000

Data Index: 401863 (The Pile)

Max Activating Token Index: 337

Text #14

Max Range: 1.4748. Min Range: -1.4748

Max Act: 0.9934. Min Act: -0.0000

Data Index: 1991064 (The Pile)

Max Activating Token Index: 897

Text #15

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.0094. Min Act: -0.0000

Data Index: 888105 (The Pile)

Max Activating Token Index: 559

Text #16

Max Range: 1.4748. Min Range: -1.4748

Max Act: 1.0129. Min Act: -0.0000

Data Index: 1796522 (The Pile)

Max Activating Token Index: 317

Text #17

Max Range: 1.4748. Min Range: -1.4748

Max Act: 0.9312. Min Act: -0.0000

Data Index: 59497 (The Pile)

Max Activating Token Index: 291

Text #18

Max Range: 1.4748. Min Range: -1.4748

Max Act: 0.9844. Min Act: -0.0000

Data Index: 121450 (The Pile)

Max Activating Token Index: 538

Text #19

Max Range: 1.4748. Min Range: -1.4748

Max Act: 0.9710. Min Act: -0.0000

Data Index: 825031 (The Pile)

Max Activating Token Index: 309