Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3738 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.5277. Min Act: -0.0000

Data Index: 1218569 (The Pile)

Max Activating Token Index: 788

Text #1

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.2918. Min Act: -0.0000

Data Index: 941475 (The Pile)

Max Activating Token Index: 835

Text #2

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.2908. Min Act: -0.0000

Data Index: 182763 (The Pile)

Max Activating Token Index: 465

Text #3

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.1493. Min Act: -0.0000

Data Index: 1449705 (The Pile)

Max Activating Token Index: 612

Text #4

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.2749. Min Act: -0.0000

Data Index: 1976463 (The Pile)

Max Activating Token Index: 339

Text #5

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.1466. Min Act: -0.0000

Data Index: 881698 (The Pile)

Max Activating Token Index: 741

Text #6

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.1879. Min Act: -0.0000

Data Index: 289024 (The Pile)

Max Activating Token Index: 532

Text #7

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.1793. Min Act: -0.0000

Data Index: 1125885 (The Pile)

Max Activating Token Index: 752

Text #8

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.0930. Min Act: -0.0000

Data Index: 1170342 (The Pile)

Max Activating Token Index: 965

Text #9

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.1400. Min Act: -0.0000

Data Index: 1493643 (The Pile)

Max Activating Token Index: 500

Text #10

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.1488. Min Act: -0.0000

Data Index: 916377 (The Pile)

Max Activating Token Index: 601

Text #11

Max Range: 1.5277. Min Range: -1.5277

Max Act: 0.9886. Min Act: -0.0000

Data Index: 1254124 (The Pile)

Max Activating Token Index: 609

Text #12

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.1363. Min Act: -0.0000

Data Index: 519038 (The Pile)

Max Activating Token Index: 295

Text #13

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.0676. Min Act: -0.0000

Data Index: 582658 (The Pile)

Max Activating Token Index: 32

Text #14

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.1049. Min Act: -0.0000

Data Index: 1809687 (The Pile)

Max Activating Token Index: 603

Text #15

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.0867. Min Act: -0.0000

Data Index: 253322 (The Pile)

Max Activating Token Index: 675

Text #16

Max Range: 1.5277. Min Range: -1.5277

Max Act: 0.9541. Min Act: -0.0000

Data Index: 1077545 (The Pile)

Max Activating Token Index: 338

Text #17

Max Range: 1.5277. Min Range: -1.5277

Max Act: 0.9997. Min Act: -0.0000

Data Index: 1965367 (The Pile)

Max Activating Token Index: 105

Text #18

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.0202. Min Act: -0.0000

Data Index: 427781 (The Pile)

Max Activating Token Index: 17

Text #19

Max Range: 1.5277. Min Range: -1.5277

Max Act: 1.0540. Min Act: -0.0000

Data Index: 265382 (The Pile)

Max Activating Token Index: 639